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Abstract 


Defence  Research  &  Development  Canada  (DRDC)  has,  for  several  years  now,  been  inves¬ 
tigating  methods  to  augment  or  enhance  existing  shipboard  Command  and  Control  System 
(CCS)  capabilities.  As  part  of  this  investigation,  llbrn  Applied  Research  Project  (ARP) 
focuses  on  the  naval  combat  resource  management  in  the  context  of  Above  Water  Warfare 
( AWW) . 

The  project,  a  summary  of  which  is  presented  in  this  report,  has  explored  concepts  con¬ 
cerned  with  the  design,  development,  implementation,  and  evaluation  of  a  computer-based, 
real-time  decision  support  system  that  could  be  integrated  into  the  future  Canadian  plat¬ 
forms  to  assist  operators  in  conducting  tactical  Command  and  Control  (C2),  particularly 
for  combat  resource  management. 

The  project  resulted  in  a  list  of  achievements,  which  includes  scientific  publications,  al¬ 
gorithms,  software  tools,  and  recommendations  for  follow-on  work.  It  has  also  provided 
inputs  to  several  projects,  including  INCOMMANDS  and  SISWS  Technology  Demonstra¬ 
tion  Projects  (TDPs). 


Resume 


Recherche  et  developpement  pour  la  defense  Canada  (RDDC)  etudie,  depuis  plusieurs 
annees,  des  methodes  pour  ameliorer  et  augmenter  les  capacites  des  systemes  de  cornman- 
dernent  et  controle  (C2)  a  bord  des  navires  de  combat.  Le  projet  de  recherche  appliquee 
(PRA)  llbrn,  sur  la  gestion  des  ressources  de  combat  dans  le  cadre  de  la  guerre  navale 
aerienne  et  de  surface,  represente  un  des  volets  de  cette  etude. 

Ce  projet,  dont  un  resume  est  presente  dans  ce  rapport,  a  explore  des  concepts  visant  la 
conception,  le  developpement,  l’execution  et  1’evaluation  des  systemes  interactifs  d’aide  a 
la  decision  informatises.  Ces  concepts  pourraient  etre  integres  aux  futures  plates-formes 
navales  canadiennes  pour  aider  des  operateurs  dans  la  conduite  des  taches  reliees  au  C2 
tactique,  en  particulier  la  gestion  des  ressources  de  combat. 

Le  projet  a  resulte  en  une  liste  de  realisations  qui  inclut  des  publications  scientifiques, 
des  algorithmes,  des  outils  logiciels  et  des  recommandations  pour  des  projets  futurs.  II  a 
egalement  contribue  a  plusieurs  projets  majeurrs,  incluant  les  demonstrateurs  technolo- 
giques  INCOMMANDS  et  SISWS. 
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Executive  summary 


Combat  Resource  Management  (llbm)  Applied  Research  Project 
(ARP):  Final  report 

A.  Benaskeur;  DRDC  Valcartier  TR  2009-300;  Defence  R&D  Canada  -  Valcartier; 
December  2009. 

Advances  in  threat  technology,  the  increasing  difficulty  and  diversity  of  open-ocean  and 
littoral  ( i.e .,  near  land)  scenarios,  and  the  volume  and  imperfect  nature  of  data  to  be 
processed  under  time-critical  conditions  pose  significant  challenges  for  future  shipboard 
Command  and  Control  Systems  (CCSs).  Among  other  functionalities,  the  CCS  provides 
capabilities  to  allow  operators  to  evaluate  the  threat  level  of  the  different  objects  within 
the  Volume  Of  Interest  (VOI),  and  use  the  shipboard  combat  resources,  when  required,  to 
response  to  them. 

Management  of  tactical  shipboard  combat  resources,  as  a  part  of  military  naval  Command 
and  Control  (C2)  process,  provides  a  real  world  application  that  involves  both  human  and 
software  decision-makers.  To  defend  itself,  a  platform,  such  as  a  Halifax  Class  Frigate,  uses 
different  systems  and  modules  that  interact  directly  or  indirectly  together.  Therefore,  it  is 
very  necessary  to  propose  ways  to  allocate  and  coordinate  the  use  of  the  different  systems 
in  order  to  increase  the  ship’s  defensive  effectiveness  against  potential  threats. 

To  achieve  this  requirement,  the  llbm  Applied  Research  Project  (ARP)  was  defined  to 
focus  on  the  naval  combat  resource  management  in  the  context  of  Above  Water  Warfare 
(AWW),  by  exploring  concepts  concerned  with  the  design,  development,  implementation, 
and  evaluation  of  a  computer-based,  real-time  decision  support  system  that  could  be  in¬ 
tegrated  into  the  future  Canadian  platforms  to  assist  operators  in  conducting  tactical  C2. 
More  specifically,  this  project  addressed  the  following  problems:  combat  resource  alloca¬ 
tion  problem,  combat  resource  coordination  problem,  tactics  generation  and  evaluation, 
and  Operations  Room  Officer  (ORO)  plan  integration. 

The  project  results  in  a  list  of  achievements,  which  includes: 

1.  DRDC  technical  reports 

2.  Open-literature  scientific  publications 

3.  PhD  and  Master  Theses 

4.  Software  tools 

5.  Knowledge  about  several  technologies  that  can  be  used  to  tackle  resource  management 
problems,  and 

6.  Finally,  inputs  to  DRDC  and  Department  of  National  Defence  (DND)  major  projects, 
such  as  INCOMMANDS  and  SISWS  Technology  Demonstration  Projects  (TPDs),  and 
Halifax  Modernization  Command  and  Control  System  (HMCCS)  acquisition  project. 

These  achievements  are  summarized  in  this  report.  Note  that  this  report  is  not  intended  to 
provide  all  the  details  about  these  achievements.  Reference  to  the  appropriate  documents, 
which  provide  further  details,  is  given. 
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Les  progres  realises  en  technologie  de  la  menace,  la  difficulty  et  la  diversite  croissantes  des 
scenarios  en  haute  mer  et  pres  du  littoral,  ainsi  que  le  volume  et  la  nature  imparfaite  des 
donnees  a  traiter  dans  des  temps  critiques  represented  des  defis  significatifs  pour  les  futurs 
systemes  de  commandement  et  controle  embarques.  En  plus  d’autres  fonctionnalites,  un 
systeme  de  commandement  et  controle  doit  pernrettre  aux  operateurs  d’evaluer  efficacement 
le  niveau  de  menace  des  differents  objets  dans  une  zone  d’interet  et,  au  besoin,  d’employer 
les  ressources  de  combat  a  bord  du  navire  pour  repondre  aux  menaces. 

La  gestion  des  ressources  tactiques  de  combat  embarquees,  comrne  partie  du  processus  C2 
naval  militaire,  definit  un  domaine  application  qui  implique  a  la  fois  des  decideurs  humains 
et  logiciels.  Pour  se  defendre,  une  plate-fornre,  telle  qu’une  fregate  de  classe  Halifax,  utilise 
differents  systemes  et  modules  qui  interagissent  directement  ou  indirectement  ensemble. 
Par  consequent,  il  devient  necessaire  de  proposer  des  manieres  d’allouer  et  coordonner 
l’utilisation  des  differents  systemes  afin  d’augmenter  l’efficacite  de  defense  du  navire  face 
aux  menaces  potentielles. 

Pour  ce  faire,  le  projet  de  recherche  appliquee  llbm  a  ete  defini  avec,  comrne  probleme 
cible,  la  gestion  des  ressources  navales  de  combat  dans  le  cadre  de  la  guerre  aerienne 
et  de  surface.  Le  projet  a  explore  des  concepts  visant  la  conception,  le  developpement, 
l’execution  et  revaluation  de  systemes  d’aide  a  la  decision  informatises.  Ces  derniers  pour- 
raient  etre  integres  dans  les  futures  plates-formes  canadiennes  pour  aider  des  operateurs 
dans  la  conduite  des  operations  liees  au  C2  tactique,  et  plus  particulierement  la  gestion  des 
ressources  de  combat. 

Plus  specifiquenrent,  ce  projet  a  aborde  les  problemes  suivants  :  l’allocation  des  ressources  de 
combat,  la  coordination  des  ressources  de  combat,  la  generation  et  1’evaluation  des  tactiques, 
et  l’integration  des  plans  pour  l’officier  de  la  salle  des  operations  (ORO,  en  anglais). 

Le  projet  a  resulte  en  un  ensemble  de  realisations,  incluant  : 

1.  Des  rapports  techniques  de  la  RDDC 

2.  Des  publications  scientifiques  dans  la  litterature  ouverte 

3.  Des  memoires  de  maitrise  et  theses  de  doctorat 

4.  Des  outils  logiciels 

5.  La  maitrise  de  plusieurs  technologies  potentiellenrent  exploitables  pour  aborder  les 
problemes  lies  a  la  gestion  des  ressources  de  combat. 

6.  Des  contributions  a  des  projets  de  la  RDDC  et  du  MDN,  tels  que  les  demonstrateurs 
technologiques  INCOMMANDS  et  SISWS,  et  le  projet  d’acquisition  HMCCS. 
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Ces  realisations  sont  brievement  decrites  dans  le  present  rapport,  sans  toutefois  presenter 
tous  les  aspects.  Des  references  aux  documents  appropries  sont  donnees  pour  plus  de  details. 
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1  Introduction 


Advances  in  threat  technology,  the  increasing  difficulty  and  diversity  of  open-ocean  and 
littoral  ( i.e .,  near  land)  scenarios,  and  the  volume  and  imperfect  nature  of  data  to  be 
processed  under  time-critical  conditions  pose  significant  challenges  for  future  shipboard 
Command  and  Control  Systems  (CCSs).  Among  other  functionalities,  the  CCS  provides 
capabilities  to  allow  operators  to  evaluate  the  threat  level  of  the  different  objects  within  the 
Volume  of  Interest  (VOI),  and  when  deemed  necessary,  use  the  shipboard  combat  resources 
to  response  to  them.  However,  current  operational  systems  generally  provide  little  support 
for  tactical  decision  making  in  complex,  highly  dynamic  scenarios  where  time  for  decision¬ 
making  and  action  execution  is  at  a  premium.  The  need  for  such  support  is  all  the  more 
pressing  given  the  current  emphasis  on  littoral  warfare,  including  asymmetrical  threats,  that 
results  in  reduced  reaction  time  and  the  need  to  deal  quickly  and  correctly  with  complex 
Rules  Of  Engagement  (ROE)  designed  to  increase  the  efficiency  of  the  ship  operations  and 
avoid  undesirable  consequences. 

Management  of  tactical  shipboard  combat  resources,  as  a  part  of  military  naval  Command 
and  Control  (C2)  process,  provides  a  real  world  application  that  involves  both  human  and 
software  decision-makers.  To  defend  itself,  a  platform,  such  as  a  Halifax  Class  Frigate,  uses 
different  systems  and  modules  that  interact  directly  or  indirectly  together.  Therefore,  it  is 
very  necessary  to  propose  ways  to  allocate  and  coordinate  the  use  of  the  different  systems 
in  order  to  increase  the  ship’s  defensive  effectiveness  against  potential  threats. 

Defence  Research  &  Development  Canada  (DRDC)  has  for  several  years  now  been  inves¬ 
tigating  methods  to  augment  or  enhance  existing  shipboard  CCS  capabilities.  As  part  of 
this  investigation,  llbrn  Applied  Research  Project  (ARP)  focuses  on  the  naval  combat  re¬ 
source  management  in  the  context  of  Above  Water  Warfare  (AWW),  by  exploring  concepts 
concerned  with  the  design,  development,  implementation,  and  evaluation  of  a  computer- 
based,  real-time  decision  support  system  that  could  be  integrated  into  the  future  Canadian 
platforms  to  assist  operators  in  conducting  tactical  C2,  particularly  for  combat  resource 
management.  More  specifically,  this  project  addressed  the  following  problems,  which  are 
described  in  more  details  in  Chapter  2. 

1.  Combat  Resource  Allocation  Problem 

2.  Combat  Resource  Coordination  Problem 

3.  Tactics  Generation  and  Evaluation 

4.  ORO  plan  integration 

The  report  is  organized  as  follows.  Chapter  2  presents  an  exhaustive  list  with  short  descrip¬ 
tion  of  the  set  of  problems  that  have  been  addressed  by  the  different  activities  conducted 
under  llbrn.  Chapter  3  summarizes  the  achievements  of  the  project.  Conclusions  and 
recommendations  are  given  in  Chapter  4. 
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2  Problems  Addressed 


The  activities  conducted  under  the  llbm  ARP  project  allowed  tackling  different  problems 
related  to  the  naval  combat  resource  management,  from  both  technological  and  human 
factor  perspectives.  The  following  sections  give  a  list  of  these  of  problems  with  short 
description.  More  details  can  be  found  in  the  reports  related  to  the  respective  tasks  and 
activities. 

2.1  Combat  Resource  Allocation 

A  resource  is  any  substance  or  (set  of)  object(s)  the  cost  or  available  quantity  of  which 
induce  some  constraint  on  the  operations  that  use  it.  For  instance,  hardkill  naval  engage¬ 
ments  require  fire  control  radars.  The  fire  control  radars  on  the  Canadian  frigates,  i.e.  the 
STIR,  can  track  and  illuminate  only  one  target  at  a  time.  Therefore,  for  the  surface-to- 
missile  and  the  57  mm  gun,  only  two  fire  control  channels  are  available,  which  constrains 
their  allocation  process. 

Resources  are  tightly  linked  with  the  notions  of  time  and  concurrency.  There  may  be 
complex  relations  that  link  the  duration  of  an  activity  with  the  amount  of  resource  it  uses, 
consumes  and/or  produces.  Therefore,  the  combat  resource  allocation  problem  boils  down 
to  two  classes  of  problems,  which  are  the  resource  allocation  planning  and  the  resource 
allocation  scheduling.  The  border  between  the  two  is  often  very  fuzzy  and  very  dependent 
on  the  targeted  problem.  The  following  definitions  aim  at  making  this  border  as  crisp  as 
possible  for  the  problem  of  interest. 

2.1.1  Resource  Allocation  Planning 

Resource  allocation  planning  is  about  assignment  of  resources  to  activities,  where  the  start 
and  end  times  of  each  activity  are  given.  In  dynamic  contexts,  this  represents  a  continuing 
process  of  analyzing  relevant  information  from  the  present  and  the  past,  and  the  assessment 
of  probable  future  developments  so  that  an  allocation  strategy  may  be  determined  that 
enables  the  overall  system  to  meet  its  stated  objectives.  In  the  military  context,  this  often 
referred  to  as  the  Weapon-Target  Assignment  (WTA)  [1] .  The  problem  consists  of  optimally 
assigning  weapons  to  targets  so  that  the  total  expected  survival  value  of  the  targets  after 
all  the  engagements  is  minimized. 

The  enormous  combinatorial  complexity  of  the  problem  implies  that,  even  with  the  su¬ 
percomputers  available  today,  optimal  solutions  cannot  be  obtained  in  real-time.  Good 
heuristics  must  therefore  be  developed  to  solve  the  problem  [2].  In  summary,  in  the  naval 
resource  management  problems,  allocation  planning  consists  of  selecting  which  weapons 
should  engage  which  threat,  independently  of  the  order  in  which  the  different  engagements 
will  actually  take  place. 
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2.1.2  Resource  Allocation  Scheduling 


Resource  allocation  scheduling  consists  in  assignment  of  start  and  end  times  to  activities, 
where  each  activity  requires  given  resources  with  given  capacities.  In  pure  scheduling  prob¬ 
lems,  activities  are  already  chosen  (or  given),  leaving  only  the  problem  of  determining  a 
feasible  order.  In  naval  operations,  scheduling  determines  when  a  specific  defensive  action  (a 
specific  weapon  against  (a)  specific  threats(s))  will  take  place.  Schedule  constraints  specify 
when  an  action  should  start  or  end  based  on  duration,  predecessors,  resource  availability, 
or  intended  interception  time. 

It  is,  however,  very  important  to  mention  that  the  allocation  of  tactical  combat  resources 
in  naval  engagements  involves  both  classes  of  problems: 

1.  Reasoning  about  limited  time  and  scarce  resources  is  at  the  very  core  of  the  tactical 
resource  management  problem,  such  as  in  purely  scheduling  problems. 

2.  Resource  management  problem  also  involves  choices.  This  problem  cannot  just  be 
confined  to  a  task  ordering,  but  includes  choices  about  which  resources  to  use  for  each 
task  (action).  For  a  given  task,  several  alternative  resources  may  be  available  that 
have  differing  cost  and/or  durations,  such  as  in  pure  allocation  planning  problems. 

Therefore,  naval  resource  allocation  problem  is  about  allocation  of  both  resources  and  start 
and  end  times  to  activities.  This  defines  a  joint  resource  allocation  planning  &  scheduling 
problem.  Note  that,  besides  the  two  above  described  fundamental  problems,  a  set  of  different 
problems  needs  to  be  addressed  within  the  combat  resource  management  context.  These 
problems  are  either  imposed  by  the  nature  of  the  primary  problem  to  be  solved  and/or  the 
environment  in  which  it  must  be  solved,  or  by  the  approaches  chosen  to  solve  it. 

2.1.3  Application  to  Hardkill/Softkill  Allocation 

Resource  allocation  planning  and  scheduling  algorithms  were  developed  and  applied  to 
the  dynamic  allocation  of  Hardkill  and  Softkill  weapons  on-board  a  single  ship.  These 
algorithms,  that  include  Cue  Generation,  Partly  Planner,  Holistic  Re-engagement  Planner, 
Holistic  Tabu  Planner,  are  described  in  more  details  in  Section  3.2.1. 

2.2  Combat  Resource  Coordination 

Given  the  differing  nature  of  the  existing  units  and  their  combat  resources,  the  effectiveness 
of  a  defensive  plan  depends  on  all  the  involved  units/resources,  as  well  as  the  environment 
and  threats  properties.  Even  though  the  optimality  of  the  individual  plans  is  assumed,  there 
still  are  coordination/cooperation  problems  that  need  to  be  solved  in  order  to  guarantee 
the  viability  and  effectiveness  of  the  entire  defence  strategy.  These  problems  are  inherent 
to  distributed  environments,  such  as  combat  resource  management  for  naval  Task  Groups, 
and  concern  interactions  within  the  decentralized  problem  solving  process.  Examples  of 
common  types  of  interactions  include: 

1.  Cooperation  -  Cooperation  defines  joint  operation  or  action,  that  is  the  process  of 
working  together  toward  a  common  goal;  sharing  effort,  expertise,  and  resources  to 
achieve  some  mutually  desirable  outcome.  This  is  about  synergy. 
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2.  Coordination  -  Coordination  is  the  process  of  managing  interactions  and  dependencies 
between  activities.  With  strictly  independent  activities,  where  there  is  no  interaction 
or  dependency,  there  is  obviously  no  need  of  coordination.  Therefore  coordination  can 
be  viewed  as  a  regulation  process  of  diverse  interacting  and/or  inter-dependent  tasks 
within  an  integrated  operation.  The  interaction  and  the  dependency  are  seldom  direct, 
but  through  shared  resources,  which  act  as  constraints  on  the  different  activities. 
This  is  why  most  of  coordination  problems  can  be  viewed  as  Constraint  Satisfaction 
Problems  (CSP). 

3.  Negotiation  -  Negotiate  is  the  process  by  which  the  different  parties  involved  come  to 
an  agreement  acceptable  to  all  of  them. 

In  naval  context,  these  interaction  problems  may  occur  on-board  a  single  ship,  as  well  as 
within  a  set  of  cooperating  platforms  ( e.g .,  Task  Group,  Coalition,  etc.).  Below  are  given 
more  detailed  definitions  and  examples  of  the  practical  coordination  and  cooperation  that 
were  tackled  within  this  project. 

2.2.1  Hardkill  &  Softkill  Coordination 

It  is  possible  to  observe  different  interactions  between  the  different  resources,  even  on¬ 
board  a  single  platform.  These  interactions  may  be  positive  (to  be  re-enforced),  negative 
(to  be  removed,  or  at  least  minimized),  or  simply  neutral.  Two  examples  of  possible  in¬ 
teractions  between  hardkill  and  softkill  weapons,  that  may  require  some  sort  of  coordina¬ 
tion/cooperation,  are  given. 

An  example  of  positive  interaction  is  given  by  the  combination  Jamming  +  Chaff.  If,  the 
probability  of  success  for  the  Chaff  alone  on  a  threat  is  p\ ,  and  for  the  Jamming  is  p2,  then 
the  use  of  the  two  together  gives  an  efficiency  ps  such  as 


1  ~P3  <  (1  -Pi)(l  ~P2)  =  P3  >  1  -  (1  -Pi)(l  ~P2) 

This  efficiency  is  superior  to  using  the  two  weapons  separately.  So  there  is  a  synergy  between 
the  two  weapons.  We  talk  about  super-additivity  of  effects. 

On  the  negative  interactions  side,  when  a  Separate  Tracking  and  Illuminating  Radar  (STIR) 
or  Close-In  Weapon  System  (CIWS)  radar  is  trying  to  guide  a  hardkill  weapon  through  a 
Chaff  cloud,  its  range  might  be  greatly  diminished.  In  fact,  the  Chaff  cloud  scrambles  the 
radars.  We  talk  about  sub-additivity  of  effects. 

Hardkill  and  Softkill  coordination  techniques  have  been  investigated  in  this  project  and  a 
set  of  capabilities  have  been  implemented  and  evaluated,  as  described  in  Section  3.2.1. 

2.2.2  Engagement  &  Ship  Position  Coordination 

During  engagements,  the  survivability  can  be  greatly  increased  if  the  ownship  is  appro¬ 
priately  positioned  with  respect  to  threats  and  the  environment.  The  position  and  the 
manoeuvres  of  the  ship  play  a  key  role  in  the  ship’s  defensive  plan.  Therefore,  the  ship 
navigation  needs  to  be  coordinated  with  the  weapon  deployment  in  order  to  increase  the 
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ship’s  survivability.  Examples  of  problems  related  to  the  coordination  of  ship  navigation 
and  positioning  are  listed  below. 

Blind  Zones  -  Since  the  effectiveness  of  a  particular  weapon  varies  depending  on  the 
orientation  of  the  ownship  with  respect  to  the  threats  faced,  a  key  element  of  the 
coordination  process  is  to  manoeuvre  the  ownship  to  most  effectively  use  all  the 
weapons  available,  that  is,  to  reduce  the  constraints  due  to  the  weapons  blind  zones. 
Figure  1  shows  a  division  of  the  environment  surrounding  the  ship  into  sectors  on 
the  basis  of  the  hardkill  and  softkill  combat  resources  available  in  each  sector.  These 
sectors  will  have  to  move  along  with  the  ownship,  and  maintain  the  same  relative 
orientation  with  respect  to  the  ownship. 


Figure  1:  Resource  Blind  Zones 

No  Fire  Zones  -  These  are  due  to  the  presence  of  other  non-hostile  platforms,  such  as 
protected  units,  friendly  ships,  neutral  vessels,  etc.  Even  though  most  the  activities 
focused  on  the  single  ship  configuration,  taking  into  account  the  no-fire  zones  will 
help  facilitate  the  generalization  of  the  developed  capabilities  to  the  multi-ship/area 
defence  operations. 

Signature  Reduction  -  Various  constraints  can  be  placed  on  the  ship  movement.  One 
example  is  given  by  the  ship’s  exposed  Radar  Cross  Section  (RCS).  This  constraint  im¬ 
poses  to  the  ownship  a  movement  that  reduces  the  RCS  exposed  to  incoming  threats, 
since  the  capability  of  threats  to  lock  onto  the  ship  is  directly  related  to  the  RCS 
of  the  ship  they  see.  Thus,  the  selection  of  appropriate  ship  positions  helps  make  it 
considerably  harder  for  threats  to  lock  and  keep  a  lock  on  the  ship.  A  simple  and 
convenient  way  to  consider  the  exposed  RCS  of  the  ownship  is  to  sub-divide  the  en¬ 
vironment  surrounding  the  ship  into  number  of  sectors,  as  in  the  case  of  blind  zones. 
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The  subdivision  is  based  here  on  the  size  of  the  exposed  in  each  sector. 

Potential  Damage  Reduction  -  In  case  of  potential  eminent  impact  on  ownship,  the 
latter  can  be  oriented  to  reduce  this  damage.  Resources  on-board  the  ship  are  of 
varying  importance,  and  the  less  important  may  be  sacrificed  to  save  the  vital  ones. 

The  above-discussed  set  of  constraints  was  used  to  coordinate  the  hardkill  and  softkill 
weapons  deployment  with  the  ownship  movement  to  maximize  their  effectiveness. 

2.2.3  Multi-Ship  Combat  Power  Coordination 

The  majority  of  work  performed  within  this  project  was  oriented  towards  issues  of  coordi¬ 
nation  of  resource  management  processes  on  a  single  platform 1 ,  as  described  in  the  previous 
section.  Due  to  the  increasing  variety  of  situations  it  can  have  to  face,  a  single  ship  is  be¬ 
coming  more  and  more  restricted  in  actions  and  responses  it  can  undertake,  making  it  more 
vulnerable.  This  restriction  can  take  different  forms:  the  ship  resource/human  capabilities, 
the  endurance,  the  reliability,  etc.  Therefore,  to  provide  an  extended  capability  and  higher 
endurance  and  reliability,  naval  forces  are  more  and  more  often  organized  into  operational 
groupings  for  specific  missions  or  tasks,  which  defined  the  concept  of  Task  Groups  that  is 
becoming  the  norm  in  today’s  naval  operations.  A  Task  Group  is  formed  by  two  or  more 
ships2,  supported  by  aircraft,  helicopters,  and/or  submarines. 

The  obvious  extension  of  the  coordination  problem  to  the  Task  Group  case  is  to  consider 
issues  of  coordination  of  combat  resources  through  a  set  of  distributed  platforms.  In  this 
case,  the  main  challenge,  compared  to  the  resource  coordination  on-board  a  single-ship, 
becomes  the  limited  communication  bandwidth  constraint.  A  comprehensive  analysis  and 
investigation  of  this  problem  was  beyond  the  scope  of  this  project.  However,  investigation 
was  performed  to  use  mobile  agent  technology  (see  Section  3.1.1  for  definition)  to  reduce 
bandwidth  utilization  during  plan  coordination  process. 

2.2.4  Engagement  &  Surveillance  Coordination 

Target  engagement  using  hardkill  weapons  cannot  start  without  a  target  re-acquisition  by 
the  Fire  Control  Radar  (FCR)  assigned  to  it.  The  FCR  initiates  its  search  within  an 
area  defined  mostly  by  the  2D  bearing-elevation  information  obtained  from  the  surveillance 
system.  Following  a  specific  pattern,  the  FCR  will  scan  this  region  of  the  Volume  Of  Interest 
(VOI)  until  it  detects  and  locks  on  the  target,  for  which  a  track  is  then  maintained.  The 
target  course  and  speed  contained  in  this  FCR  track  is  then  used  to  compute  a  Predicted 
Intercept  Point  (PIP)  inside  the  weapon  engagement  envelope.  The  goal  is  to  provide 
guidance  (for  the  missile)  or  the  pointing  (for  the  gun)  information  toward  the  engaged 
threat.  During  this  threat  re- acquisition 3  phase,  the  FCR  has  a  search  time  that  depends 
on  several  factors,  such  as:  the  ownship  weapons  properties,  Command  &  Control  System 
(CCS)  performance,  the  operator  skill/training,  the  attacking  target  characteristics,  etc. 

1.  Also  referred  to  as  point  or  self-defence  operations. 

2.  e.g.,  destroyers,  frigates. 

3.  Also  referred  to  as  search  and  lock-on. 
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The  quality  of  the  information  handed-over  determines  the  volume  the  FCR  must  scan.  The 
duration  of  the  search  and  lock-on  operation  depends  on  the  volume  to  be  scanned  and  the 
detection  probability  of  the  FCR.  This  duration  is  subtracted  from  the  total  reaction  time 
available  to  the  decision-maker  and  necessary  to  the  ship  survival.  A  poor  quality  tactical 
picture  causes  the  FCR  to  search  in  a  large  volume,  which  normally  will  take  more  time  to 
re-acquire  the  target  and  which  may  have  grave  consequences  on  the  ownship  reaction  time. 
Therefore,  the  search  and  lock-on  duration  should  be  limited  depending  on  the  available 
reaction  time.  This  is  equivalent  to  limit  the  uncertainty  contained  in  the  tracks  provided  by 
the  surveillance  system,  by  executing  actions  for  information  gathering  (sensor  allocation) 
or  information  processing  (data  fusion  adaptation). 

2.2.5  Hardkill/Softkill  Prioritization 

Considering  the  tactical  situation  involving  a  ship  defending  itself  against  a  set  of  incoming 
threats,  engageability  assessment  is  used  to  reduce  planning  complexity  and  reaction  time. 
This  concept  of  engageability  assessment  is  exploited  to  prioritize  the  use  of  the  softkill 
defence  strategy  over  the  hardkill  one,  when  the  latter  are  declared  non-feasible.  Hence, 
given  the  quality  Q  of  the  tracks  handed-over  to  the  Fire  Control  Radar,  the  estimation  of 
the  search  and  lock-on  duration  s,  and  threat /weapons  characteristics,  the  combat  resource 
manager  can  establish  the  feasibility  of  hardkill  engagements. 

A  non-feasibility  assessment  simplifies  the  response  planning  process  by  discarding  all  the 
options  that  involve  hardkill  weapons.  In  such  situations,  softkill  strategies  are  advocated, 
and  priority  is  granted  to  their  use  against  non-engageable  targets,  since  their  deployment 
does  not  require  FCR  support  and  their  effectiveness  is  not  affected  by  the  quality  of  the 
tactical  picture.  Such  a  prioritization  of  weapons  assignment  prevents  wasting  reaction 
time,  by  eliminating  unfeasible  assignments,  and  helps  increase  the  ship  survivability  by 
maximizing  the  number  of  engaged  threats.  However,  when  hardkill  engagements  are  judged 
to  be  feasible,  further  decisions,  involving  more  criteria,  will  be  required  to  select  between 
hardkill  strategy,  softkill  strategy,  or  combination  of  both4. 

2.2.6  Sensor  Management 

The  objective  of  any  surveillance  mission  is  to  gather  information  about  the  presence  and 
activity  of  all  potential  targets  in  a  volume  of  interest.  Surveillance  is  the  systematic  ob¬ 
servation  of  a  tactical  situation  by  sensors.  The  data  collected  by  military  surveillance  is 
used  by  analysts,  both  human  and  software,  to  build  a  representation  of  the  tactical  sit¬ 
uation.  This  representation  may  describe  in  detail  the  environment  (terrain,  weather  and 
any  man-made  structures),  the  spatial  coordinates  of  friendly,  enemy  or  neutral  targets, 
and  may  include  temporal  changes  if  the  observation  period  is  sufficiently  long  in  duration. 
Information  gathering  process  performance  can  be  improved  with  the  application  of  sen¬ 
sor  management.  Sensor  management  is  the  coordination  and  control  of  limited  sensing 
resources  to  collect  the  most  complete  and  accurate  data  observed  from  a  dynamic  scene. 
Sensor  management  is  a  key  enabling  element  of  tactical  surveillance. 

4.  As  addressed  by  combat  resource  allocation  and  coordination  activities. 
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This  section  will  discuss  some  of  the  issues  that  sensor  management  should  address  in  a 
tactical  surveillance  situation,  some  of  the  emerging  approaches  to  sensor  management,  the 
architecture  in  which  sensor  management  is  applied  to  control  sensing  resources,  and  some 
of  the  direct  issues  associated  with  information  gathering. 

Potential  issues  in  tactical  surveillance,  for  which  strategies  are  required,  would  include: 

•  If  a  sensing  resource  has  multiple  operating  modes,  then  the  sensor  management  should 
make  use  of  the  most  optimal  mode  for  the  task  being  done  provided  that  there  is  no 
other  overriding  reason  not  to.  The  performance  of  certain  sensors  is  dependent  on  the 
position  and  orientation  relative  to  the  VOI  that  they  are  observing.  Although  the  sensor 
management  system  does  not  have  control  of  the  host  platform’s  navigation,  it  can  make 
a  request  to  the  navigation  control  module  to  position  or  orient  the  platform  so  that  it 
can  improve  the  quality  of  the  data  stream. 

•  If  a  sensing  resource  when  in  operation  is  in  conflict  with  another  resource  (another  type 
of  sensor  or  a  weapon)  then  the  management  must  determine  which  action  or  data  stream 
is  more  important  and  prevent  the  other  from  operating  or  must  allow  for  some  schedule 
to  allow  one  sensor  to  be  used  for  a  period  of  time  and  then  the  other. 

•  If  a  surveillance  mission  is  being  undertaken  by  a  group  (more  than  one  platform),  then 
the  sensor  management  must  be  capable  of  ensuring  that  the  sensors  from  different  plat¬ 
forms  and  the  platforms  themselves  do  not  compete  with  each  other  but  rather  cooperate. 
Two  such  situations  are:  the  sensors  for  one  platform  may  have  their  view  of  the  VOI 
occluded  by  another  platform,  and  the  sensors  from  one  platform  may  conflict  with  those 
of  another. 

•  While  changing  sensor  mode,  the  data  stream  may  be  halted  during  the  period  of  tran¬ 
sition.  The  sensor  management  must  address  whether  it  is  more  important  to  maintain 
operation  in  possibly  a  sub-optimal  mode  while  maintaining  a  live  data  stream  or  to 
change  to  a  more  optimal  mode  sacrificing  continuity  of  data. 

•  While  operating  active  sensing  resources  to  monitor  the  VOI,  it  is  emitting  some  form  of 
energy.  Is  it  a  high  enough  level  and  is  it  active  for  a  long  enough  period  of  time  so  as 
to  compromise  the  security  of  that  resource?  The  sensor  management  system  must  trade 
off  the  gathering  of  more  complete  information  over  the  self  security  of  the  resource. 

•  If  a  sensing  resource  is  disabled  or  diminished  in  capability  then  the  sensor  management 
must  alter  the  sensing  allocations  and  schedule.  The  sensing  load  could  possibly  be 
redistributed  to  other  sensing  resources  that  are  capable  of  returning  a  similar  data 
stream. 

•  If  situation  changes  or  a  new  surveillance  request  is  made  to  the  system,  then  the  sensor 
management  must  address  when  and  how  to  make  the  necessary  changes.  The  sensor 
management  must  determine  if  the  surveillance  task  being  executed  should  be  allowed  to 
finish,  should  it  be  suspended,  or  terminated. 

This  is  not  an  exhaustive  list  of  issues  that  sensor  management  of  sensing  resources  in  a 
tactical  surveillance  activity  will  have  to  address  but  it  shows  that  sensor  management  is 
more  than  just  direct  control  of  sensors. 

Conscious  of  the  important  role  that  sensor  management  has  to  play  in  modern  Command 
and  Control  systems,  activities  have  been  conducted,  within  the  llbrn  project,  to: 
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1 .  Characterize  the  different  sensor  management  problems  and  the  requirements  for  their 
solution. 

2.  Demonstrate  the  benefits  that  can  be  brought  by  the  dynamic  management  of  the 
available  naval  sensing  resources. 

2.3  Tactics  Generation  and  Evaluation 

The  problem  addressed  by  this  activity  was  to  investigate  techniques  and  build  a  capability 
that  allow  the  generation  of  new  defence  tactics  and  the  evaluation  of  the  existing  ones, 
for  possible  improvements.  Even  though  the  core  problem  remains  the  same  as  the  one 
described  previously,  that  is  combat  resource  allocation  and  coordination,  there  is  still  a 
major  difference.  The  latter  is  an  off-line  planning  problem,  which  removes  the  real-time 
constraints  and  allows  the  use  of  computationally  heavier  approaches. 

This  activity  is  concerned  with  getting  the  “best”  solution  taking  into  consideration  different 
aspects  of  the  problem.  On  one  hand,  not  only  the  immediate  consequences  are  concerned, 
but  also  the  impact  of  this  decision  on  the  whole  planning  horizon.  This  is  known  as 
“multi-stage”  aspect  of  the  problem.  On  the  other  hand,  to  choose  a  decision,  a  set  of 
conflicting  decision-criteria  in  the  evaluation  of  alternatives  are  considered.  This  defines 
the  multi-criteria  (see  Section  3.1.2)  aspect  of  the  problem. 

2.4  ORO  Plan  Integration 

To  exercise  its  role  in  the  detection  and  tracking  of  ships,  aircraft  and  submarines  within  its 
area  of  interest,  the  HALIFAX  Class  frigate  relies  on  its  team  of  operators  in  the  Operations 
(Ops)  Room.  This  team  consists  of  20  or  more  people,  normally  coordinated  by,  and 
reporting  to,  the  Operations  Room  Officer  (ORO). 

OROs’  planning  responsibilities  range  from  short  term,  reactive  planning  to  long  term, 
and  deliberative  planning.  There  is  a  grey  line  between  when  the  implementation  of  set 
tactics  and  procedures  stops  and  reactive  planning  starts.  Short  term  planning  is  generally 
required  when  the  situation  dictates  a  change  in  the  current  course  of  directed  action  in 
a  manner  that  was  previously  unidentified  or  unaccounted  for.  For  short  term  planning 
the  ORO  will  typically  employ  only  those  planning  tools  that  are  readily  available  in  Ops, 
such  as  publications,  command  instructions  such  as  OPTASK  messages,  charts  and  CCS 
or  Global  Command  and  Control  System  -  Maritime  (GCCS-M)  displays. 

It  is  of  no  surprise  that  the  cognitive  work  demands  of  the  ORO  are  considerable.  From 
the  time  of  preparing  for  the  watch  through  to  its  end,  the  ORO  is  faced  with  dozens  of 
decisions,  some  minor  and  trivial  and  others  significant  in  considerations  and  repercussions. 
Hence  the  importance  of  analyzing  aspects  of  the  perceptual,  cognitive,  meta-cognitive  and 
collaborative  work  demands  on  the  ORO  of  a  HALIFAX  Class  frigate  in  conducting  tactical 
planning  and  response  management  within  and  across  the  air,  surface  and  subsurface  war¬ 
fare  areas.  The  understanding  of  ORO  demands  will  support  the  development  of  advanced 
decision  aids  tools. 
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3  Summary  of  Achievements 


This  section  provides  a  summary  of  the  achievements  resulting  from  the  different  activities 
conducted  under  the  llbrn  project.  These  include: 

1.  Investigation  of  several  cutting-edge  technological  and  human  factor  approaches,  and 
acquisition  of  knowledge  with  respect  to  their  usefulness. 

2.  Publications  in  the  form  of  DRDC  technical  reports,  DRDC  contract  reports,  open 
literature  articles,  and  university  theses. 

3.  Capabilities  in  the  form  of  simulation  environment  and  algorithms. 

4.  Input  to  major  projects,  such  as  HMCSS,  INCOMMANDS  TDP,  and  SISWS  TDP. 
These  achievements  are  described  in  more  details  in  the  following  sections. 

3.1  Investigated  Technologies 

Different  technologies  have  been  investigated  to  address  the  naval  combat  resource  man¬ 
agement  problem.  Below  is  given  a  list  and  description  of  the  ones  that  have  been  deemed 
most  relevant  to  the  problem. 

3.1.1  Agent  Technology 

Agent  has  been  a  central  technology  within  this  project.  The  agent  technology  aims  to 
conceive  entities  capable  of  acting  in  a  rational  way  via  approaches  turning  around  plan¬ 
ning,  uncertainty  reasoning,  decision  theory,  machine  learning,  vision  and  perception,  etc. 
However,  in  many  applications,  the  agent  alone  is  insufficient  to  do  all  the  tasks,  and  it  is 
preferable  to  view  it  evolving  with  other  agents.  This  defines  multi-agent  systems.  In  this 
kind  of  systems,  the  agents  interact  together  in  order  to  cooperate,  compete  or  more  simply, 
coexist.  Agents  and  multi-agent  systems  represent  a  new  way  of  analyzing,  designing,  and 
implementing  complex  systems.  The  following  summarizes  the  definition  of  agent  as  viewed 
in  this  project. 

An  agent  is  a  software  component,  situated  in  some  environment,  that  is  capable  of  flexible 
autonomous  action  in  order  to  meet  its  design  objectives  [3].  Situatedness  means  that 
the  agent  receives  sensory  input  from  its  environment  and  that  it  can  perform  actions  that 
change  the  environment  in  some  way.  Autonomy  is  used  in  the  sense  that  the  system  should 
be  able  to  act  rationally  without  the  direct  intervention  of  humans  (or  other  agents) ,  and 
that  it  should  have  control  over  its  own  actions  and  internal  state.  By  flexibility  it  is  meant 
responsive5  and  pro- activity 6 . 

The  following  variants  and  extensions  to  the  basic  paradigm  of  agent  have  been  explored 
and  used  within  this  project. 

5.  Agents  should  respond  in  timely  fashion  to  changes  in  their  environment. 

6.  Agents  should  not  simply  act  in  response  to  their  environment;  they  should  be  able  to  exhibit  oppor¬ 
tunistic,  goal-directed  behaviour  and  take  the  initiative  where  appropriate 
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1.  BDI  Agents  -  The  agents,  used  in  this  project,  model  reasoning  according  to  the 
Belief  Desire  Intention  (BDI)  model  of  Artificial  Intelligence  (AI).  Following  the  BDI 
model,  intelligent  agents  are  autonomous  software  components  that  have  explicit  goals 
to  achieve  or  events  to  handle  (Desires).  To  describe  how  they  should  achieve  these 
desires,  BDI  agents  are  programmed  with  a  set  of  plans.  Each  plan  describes  how  to 
achieve  a  goal  under  varying  circumstances.  Set  to  work,  the  agent  pursues  its  given 
goals  (Desires),  adopting  the  appropriate  plans  (Intentions)  according  to  its  current 
set  of  data  (Beliefs)  about  the  state  of  the  world. 

2.  Mobile  Agents  -  The  mobile  agent  paradigm  consists  of  small  programs  that  may 
be  dispatched  from  a  local  location  (computer)  and  transported  to  a  remote  location 
(computer)  for  execution.  There  are  several  motivations  for  using  the  mobile  agent 
approach.  Within  this  project,  the  two  major  driving  motivations  have  been  the 
reduction  of  network  traffic7  and  exploitation  of  asynchronous  interaction8,  both 
during  plan  coordination  in  multi-platform  configurations. 

3.  Multi- Agent  Systems  -  Multi- Agent  Systems  are  concerned  with  the  behaviour  of 
a  collection  of  autonomous  agents  aiming  at  solving  a  given  problem.  A  Multi- Agent 
System  can  be  defined  as  a  loosely  coupled  network  of  problem  solvers  that  work 
together  to  solve  problems  that  are  beyond  the  individual  capabilities  or  knowledge 
of  each  problem  solver.  These  problem  solvers  (agents)  are  autonomous  and  may 
be  heterogeneous  in  nature.  The  main  characteristics  of  Multi-Agent  Systems  are: 
i)  each  agent  has  incomplete  information  (or  capabilities)  to  solve  the  problem;  ii) 
there  is  no  global  system  control;  iii)  data  is  decentralized;  and  iv)  computation  is 
asynchronous. 

A  military  C2  system  is  a  multi-agent  organization  in  which  the  decision  agents  are 
both  human  and  software  decision-makers  (autonomous  agents).  The  decision  agents 
are  geographically  dispersed  due  to  the  operation  environment,  the  nature  of  sen¬ 
sors,  and  physics  and  speed  of  the  weapons.  Thus,  both  geography  and  security 
contribute  to  the  distributed  architecture  of  C2  systems.  Cooperation,  coordination 
and  communication  between  the  decision  agents  are  thus  crucial  in  such  a  distributed 
C2  architecture.  In  this  project,  the  multi-agent  technology  has  been  used  mainly 
to  decompose  the  problem  into  sub-problems  and  to  coordinate  the  sub-  problems 
solutions  using  coordination  techniques.  This  technology  enables  to  diminish  greatly 
the  planning  time  for  the  target  real-time  application  problem,  while  generating  a 
near-optimal  solution. 

4.  Holonic  Control  Systems  -  The  control  and  coordination  are  the  major  challenges 
facing  the  above  discussed  multi- agent  systems.  Theoretically,  an  ideal  solution  to  this 
control  problem  is  provided  by  a  centralized  control  strategy.  Nevertheless,  in  real 
world  applications,  the  hierarchical  architecture  has  received  much  attention  in  order 
to  overcome  centralized  control  drawbacks9.  Typical  hierarchical  control  architecture 

7.  Since  there  are  usually  several  information  flows  between  platform  computers  to  perform  even  a  simple 
task. 

8.  The  local  computer  sends  his  agent  to  a  remote  computer  to  perform  several  tasks,  and  only  when 
the  agent  finishes  does  it  retract  back  to  the  local  computer.  A  connection  does  not  need  to  be  maintained 
during  the  task  achievement. 

9.  Massive  computation  at  the  central  node;  high  requirement  for  inter-node  communications;  and  lacks 
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is  composed  of  a  set  of  controllers  structured  in  a  pyramidal  form,  where  each  distinct 
level  addresses  a  certain  class  of  problems  and  has  a  certain  level  of  authority.  In  such 
a  configuration,  the  control  and  coordination  mechanisms  follow  typically  a  master 
-  slave  relationship.  Nevertheless,  because  of  the  static  and  deterministic  nature  of 
hierarchical  control  architecture,  it  is  often  difficult  to  make  it  adaptable  to  cope  with 
unpredictable  changes,  essential  in  highly  dynamic  and  uncertain  systems,  such  as 
military  applications. 

The  heterarchical  (fully  distributed)  control  architecture  was  proposed  to  overcome 
the  limitation  of  the  hierarchical  control.  This  is  a  highly  distributed  control  form  im¬ 
plemented  by  independent,  but  cooperating,  processes  without  a  centralized  control. 
In  such  architecture,  the  control  decisions  are  obtained  by  mutual  agreement,  through 
negotiation.  The  high  flexibility,  the  reliability  -in  the  sense  of  failure  tolerance-,  the 
ease  of  implementation,  and  the  less  computational  requirement,  are  some  among  the 
most  relevant  properties  offered  by  the  heterarchical  control.  Nevertheless,  uncer¬ 
tainty,  unpredictable  target  behaviour,  and  the  large-scale  nature  of  the  distributed 
military  resource  management  problem  may  render  the  heterarchical  approach  ineffi¬ 
cient.  The  system  behaviour  under  heterarchical  control  may,  under  some  conditions, 
become  very  unpredictable  and  almost  chaotic,  which  is  unacceptable  in  military 
applications.  This  is  why  an  in  between  solution,  given  by  the  “holonic  control” 
paradigm,  represents  a  good  comprise  to  tackle  military  applications  in  general,  and 
the  naval  resource  control  and  coordination  problem  particularly.  Holonic  architec¬ 
tures  correspond  to  a  mixture  of  the  heterarchical  control  and  hierarchical  control, 
where  higher-level  holons  10  act  as  coordinator  for  lower  level  holons.  Holonic  con¬ 
trol  structures  are  the  result  of  a  marriage  between  Koestler’s  [4]  general  philosophy 
(of  living  organisms  and  social  organizations)  and  emerging  software  approaches  (dis¬ 
tributed  artificial  intelligence). 

5.  Jack  Agent-based  Language  -  The  JACK  programming  language  has  been  used 
in  this  project.  JACK  provides  an  environment  for  building,  running  and  integrat¬ 
ing  commercial-grade  multi-agent  systems  using  a  component-based  approach.  The 
JACK  agent  extends  Java  language  with  agent-oriented  concepts,  such  as  agents, 
capabilities,  events,  plans,  agent’s  knowledge  bases  (databases),  and  resource  and 
concurrency  management.  The  agents  used  in  JACK  are  intelligent  agents  based  on 
the  above  described  BDI  model.  Also,  JACK  agents  can  exhibit  reasoning  behaviour 
under  both  deliberative  (goal  directed)  and  reactive  (event  driven)  stimuli.  Each 
agent  has  i)  a  set  of  beliefs  about  the  world  (its  data  set);  ii)  a  set  of  events  that  it 
will  respond  to,  iii)  a  set  of  goals  that  it  may  desire  to  achieve  (either  at  the  request 
of  an  external  agent,  as  a  consequence  of  an  event,  or  when  one  or  more  of  its  beliefs 
change),  and  iv)  a  set  of  plans  that  describe  how  it  can  handle  the  goals  or  events 
that  may  arise.  When  an  agent  is  instantiated  in  a  system,  it  will  wait  until  it  is 

flexibility  and  robustness,  since  central  node  must  be  kept  intact. 

10.  Holons  can  be  thought  of  as  self-contained  wholes  looking  towards  the  subordinate  level  and  dependent 
parts  looking  upward.  Holons  have  a  certain  degree  of  autonomy  that  allows  them  to  make  decisions  of 
limited  scope.  The  decisions  that  a  holon  can  make  are  limited  to  accepting  the  request  being  made  and 
executing  the  request  by  utilizing  available  resources.  The  process  used  to  arrive  at  a  decision  is  only  as 
complex  as  necessary  for  that  class  of  holons  and  its  level  within  the  holarchy. 
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given  a  goal  to  achieve  or  experiences  an  event  that  it  must  respond  to.  When  such 
a  goal  or  event  arises,  it  determines  what  course  of  action  it  will  take.  If  the  agent 
already  believes  that  the  goal  or  event  has  been  handled  (as  may  happen  when  it  is 
asked  to  do  something  that  it  believes  has  already  been  achieved),  it  does  nothing. 
Otherwise,  it  looks  through  its  existing  plans  to  find  the  ones  that  are  relevant  to  the 
request  and  applicable  to  the  situation.  It  select  the  most  appropriate  for  execution, 
and  if  it  has  any  problems  executing  it,  it  looks  again  for  others  that  might  apply  and 
keeps  cycling  through  the  set  of  alternatives  until  it  succeeds  or  all  alternatives  are 
exhausted. 

3.1.2  Planning 

Several  planning  approaches  have  been  used  to  address  the  naval  combat  resource  manage¬ 
ment  problem,  as  described  below. 

1.  Reactive  Planning  -  consists  of  reflex  planner  (possibly  with  internal  state)  that 
can  be  implemented  with  any  variety  of  representations  for  condition- action  rules. 
Reactive  planners  react  to  the  environment,  without  reasoning  about  it.  This  consists 
of  predefining  a  set  of  rules  to  accomplish  a  certain  limited  goal.  A  reactive  plan 
represents  a  set  of  tests  and  reactions  able  to  handle  contingencies.  Since  each  response 
is  less  carefully  analyzed  than  in  the  deliberation-based  case  (see  definition  below), 
the  response  does  not  need  to  embody  a  complete  solution  to  the  final  goal  but  can 
merely  be  an  action  to  stabilize  the  situation  and  allow  the  time  for  elaborating  a 
more  comprehensive  plan. 

2.  Deliberative  Planning  -  consists  of  designing,  before  execution,  a  sequence  of  steps 
to  achieve  a  particular  goal.  Among  the  strengths  of  the  deliberation  is  the  fact  that 
plans  can  be  built  to  have  a  set  of  desirable  global  properties  regarding  the  goals  to 
be  attained  and  the  resources  available.  The  side  effects  of  the  actions  to  be  executed 
as  part  of  the  plan  can  be  carefully  taken  into  account  and  analyzed  before  execution 
begins.  These  properties  are  achieved  by  taking  into  account  complete  descriptions 
of  the  states  of  the  world  as  the  planner  predicts  them.  Of  course,  these  states  may 
conform  to  reality  only  if  the  environment  behaves  according  to  the  model  that  the 
planner  has  about  it.  The  more  incomplete  this  model  is  the  more  uncertainty  in  the 
behaviour  of  the  environment,  and  the  more  uncertainty  about  the  actual  states  that 
will  be  encountered  during  the  plan  execution. 

3.  Anytime  Planning  -  We  used  two  types  of  anytime/deliberative  planners  in  our 
project;  Markov  Decision  Processes  [5]  (MDPs),  and  a  Tabu  algorithm  [6].  A  Tabu 
search  is  a  local  search  approach,  while  a  standard  MDP  algorithm  is  optimal.  Thus, 
one  can  never  know  in  real-time  if  the  solution  of  the  Tabu  algorithm  is  optimal. 
This  algorithm  provides  better  results  than  a  reactive  approach  but  is  more  complex. 
Indeed,  it  may  never  generate  a  worst  plan  than  a  reactive  approach  since  the  initial 
plan  for  the  Tabu  search  is  the  one  generated  by  a  reactive  approach. 

An  optimal  solution  for  an  MDP  approach  is  too  complex  to  generate  for  our  real¬ 
time  application.  Thus  approximations  and  heuristics  have  been  developed.  We 
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also  developed  approximation  and  heuristics  to  solve  the  Weapon  Target  Assignment 
(WTA)  using,  among  others,  piecewise  linear  programming  [7]. 

4.  Markov  Decision  Process  -  Markov  Decision  Processes  (MDPs)  have  been  recently 
used  in  the  project.  First  of  all,  the  optimal  acyclic  decomposition  and  approximate 
on-line  decomposition  algorithms  produced  great  results  [5].  This  research  opened 
the  door  for  Multi-agent  Task  Associated  MDPs  (MTAMDPs)  which  plan  for  each 
resources  and  coordinated  the  plan  using  an  efficient  algorithm  which  considers  simul¬ 
taneous  actions  and  interactions  between  resources.  MDPs  prove  to  be  very  useful 
for  the  resource  allocation  problem,  and  can,  through  the  policy  iteration  approach, 
provide  an  anytime  solution  to  the  planning  problem. 

5.  Multi-Criteria  Decision  Analysis  -  Most  real  world  decision-making  problems 
require  that  multiple  and  conflicting  criteria  (points  of  view)  should  be  taken  into 
account  in  order  to  make  a  decision  which  reports  better  the  real  world  situation. 
In  particular,  decision  making  situations  often  include  a  full  range  of  social,  environ¬ 
mental,  technical,  economic,  and  financial  criteria.  For  such  situations,  reaching  the 
optimal  solution  that  maximizes  all  the  criteria  is  rarely  possible.  The  best  compro¬ 
mise  solutions  are  rather  focused  on.  Several  Multi-Criteria  Decision  Aid  (MCDA) 
methods  have  been  proposed  as  solving  approaches  to  various  types  of  multi-criteria 
problems.  Those  MCDA  methods  can  be  divided  into  two  broad  categories:  Multi- 
Attribute  utility  methods  and  Outranking  methods.  Methods  falling  within  these  two 
categories  consider  the  decision  maker  preferences,  which  are  introduced  a  priori. 

(a)  Multi-attribute  utility  methods  aggregate  local  preferences  (at  each  at¬ 
tribute’s  level)  into  a  unique  (utility,  value)  function  which  is  then  optimized. 

(b)  Outranking  methods  consist  in  building  binary  relations  (called  outranking 
relations)  in  order  to  represent  the  decision-maker’s  preferences  (based  on  the 
available  information).  For  some  outranking  methods,  discrimination  (indiffer¬ 
ence,  preference)  thresholds  and  even  veto  thresholds,  at  each  criterion  level, 
must  be  introduced  to  model  the  decision-maker’s  preferences  locally.  Usually, 
these  relations  are  neither  transitive  nor  complete.  Outranking  relations  are 
then  exploited  in  order  to  formulate  a  recommendation  that  can  solve  the  deci¬ 
sion  problem. 

3.1.3  Plan  Coordination 

Coordination  techniques  have  been  used  in  two  aspects  of  the  projects.  First,  we  coordinated 
the  hardkill  and  softkill  weapons  [8] .  In  this  work,  we  concluded  that  a  ‘central  coordinator’ 
technique  is  the  most  efficient  considering  our  real-time  environment.  Furthermore,  we 
used  coordination  mechanisms  to  coordinate  a  frigate  fleet  [9].  The  most  important  result 
demonstrates  that  the  Contract-Net  is  the  most  efficient  coordination  technique  considering 
frigate  survival  and  the  number  of  resource  used.  [10]  proposed  a  coordination  technique 
based  on  the  decomposition  of  the  problem  on  the  basis  of  the  longest  possible  engagement, 
that  is,  the  Jammer.  This  way,  the  frigate  can  elaborate  separate  plans,  which  are  much 
less  complex  than  a  whole  global  plan. 
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3.1.4  Machine  Learning 


Machine  learning11  has  been  used  in  our  resource  optimization  movement  algorithms  [11, 
10,  12].  It  has  been  used  to  learn  the  effectiveness  of  the  six  different  sectors  surrounding 
the  frigate.  These  sectors  are  defined  according  to  the  blind  zones  of  our  resources. 

3.2  Capabilities 

This  section  presents  the  capabilities  that  were  developed  during  the  llbrn  project. 

3.2.1  Algorithms 

1.  Cue  Generation  The  cue  generation  (GENCUE)  algorithm  produces  a  list  of  first 
engagements  used  by  the  different  others  planners  (presented  below).  Therein,  an 
engagement  is  characterized  by  a  defence  resource,  an  illuminator  (in  the  case  of  the 
Surface-to-Air  Missile  (SAM)  or  the  57mm  gun),  a  target  threat-level,  and  four  time- 
stamped  actions  or  cues,  namely:  (1)  Search  and  lock  on  target;  (2)  Fire;  (3)  Target 
intercept;  (4)  Kill  assessment  (i.e.,  target  destroyed  or  not). 

GENCUE  constructs  a  list  of  feasible  hardkill  engagements  starting  with  the  weapon 
that  has  the  earliest  target  intercept  time  to  the  latest.  It  starts  with  an  initial  matrix 
of  feasible  time  intervals  and  stops  when  all  time  intervals  in  the  matrix  are  empty. 
The  selection  of  the  latest  feasible  time  to  start  an  engagement  is  usually  good  practice 
in  single-target  engagements,  as  the  probability  of  interception  for  a  defensive  weapon 
typically  increases  when  the  distance  between  the  target  and  the  ownship  decreases. 
As  the  algorithm  unfolds,  a  tree  is  generated,  because  many  different  engagements 
may  be  possible  at  a  given  point  through  the  choice  of  a  particular  weapon  to  fire 
or  a  particular  threat  to  shoot  at.  A  new  branch  is  thus  generated  for  each  possible 
engagement;  the  backward  search  is  then  applied  in  a  recursive  manner  along  each 
branch,  until  no  feasible  engagements  may  be  found.  The  backward  search  is  designed 
to  maximize  the  number  of  re-engagements  of  a  target. 

2.  Partly  Planner  -  The  Partly  Planner  uses  very  low-level  reasoning  techniques  in 
order  to  elaborate  a  response  to  a  situation  in  a  very  short  reaction  time.  This  is 
very  important  in  combat  resource  management  context  because  defending  ownship 
brings  a  very  hard  and  usually  very  short  time  constraint.  For  this  planning  mode, 
the  planning  agent  maintains  a  list  of  the  objects  within  the  volume  of  interest.  These 
objects  are  sorted  (from  the  most  to  the  least  threatening)  based  on  several  criteria. 
For  this  implementation,  threat  evaluation  considers  only  the  Closest  Point  of  Ap¬ 
proach  (CPA)  of  the  threat  to  the  ownship,  and  the  time  for  the  threat  to  reach  CPA. 
Then,  the  planning  agent  applies  some  predefined  rules  for  allocating  the  resources. 
These  predefined  rules  are 

(a)  Allocate  a  SAM  and  the  57  mm  gun  to  the  most  threatening  target. 

(b)  Allocate  a  SAM  to  the  second  most  threatening  target. 

11.  Approach  that  allow  problem  solving  process  to  learn  from  past  data  and  experiences. 
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(c)  Allocate  the  Close-In  Weapon  System  (CIWS)  to  all  targets  that  enter  into  the 
CIWS  range  (one  target  at  a  time). 

The  first  two  rules  are  inspired  by  the  fact  that  there  are  only  two  STIRs  available, 
which  must  be  used  in  conjunction  with  the  SAM  and  the  57  mm  gun.  Although  rules 
used  by  the  planner  are  simple,  they  allow  all  available  resources  to  be  used  in  an 
efficient  way.  Unfortunately,  the  SAM  and  the  57  mm  gun  are  only  allocated  at  a  given 
point  in  time  to  the  two  most  threatening  objects,  and  all  others  in  the  list  (if  any) 
are  not  considered  by  this  specific  planner  (this  is  why  it  is  called  “Partly”).  In  the 
case  where  a  kill  assessment  indicates  that  an  engaged  target  has  been  destroyed,  the 
resources  that  have  been  allocated  to  it  become  available  for  the  next  most  threatening 
object  in  the  list. 

3.  Holistic  Re-engagement  Planner  -  As  it  name  indicates,  this  planner  considers 
all  targets  within  the  volume  of  interest  and  can  re-engage  with  a  SAM.  A  first  list  of 
engagements  is  obtained  with  the  cue  generation  algorithm  as  discussed  previously. 
However,  there  is  a  key  difference  in  the  SAM  engagements  from  the  cues  generated 
for  the  Partly  Planner.  Instead  of  inserting  one  SAM  engagement  at  the  latest  pos¬ 
sible  time,  it  can  use  re-engagements  of  the  SAM  against  a  target.  If  after  the  kill 
assessment  of  the  first  SAM  the  target  is  still  alive,  a  second  SAM  can  be  engaged 
to  destroy  it.  Usually  there  will  be  at  least  one  re-engagement  for  each  target.  In 
the  planning  process,  the  engagements  of  the  SAM  are  scheduled  backwards  in  time 
from  the  latest  time  of  fire  possible.  SAMs  are  added  in  the  Holistic  Re-engagement 
Planner  until  it  is  not  possible  anymore  to  do  the  kill  assessment  of  the  current  SAM 
and  to  be  able  to  engage  the  next  one.  Another  difference  from  the  Partly  Planner  is 
that  the  Holistic  Re-engagement  Planner  considers  all  targets  to  make  a  plan. 

This  planner  views  all  the  detected  threats  constituting  a  complex  organization  sur¬ 
rounding  the  ownship.  It  works  as  follows:  a  decision  tree  is  first  produced  that 
explicitly  considers,  in  a  probabilistic  manner,  all  possible  outcomes  of  a  particular 
action.  Such  a  tree  reflects,  in  fact,  a  plan  with  different  conditional  branches.  The 
conditional  branches  allow  taking  into  account  results  of  actions.  For  instance,  during 
the  plan  execution,  one  should  follow  one  branch  or  another  depending  on  the  result  of 
an  engagement  to  some  threat  T{i).  If  this  engagement  has  succeeded,  then  the  plan 
continues  by  following  a  branch  where  it  does  not  consider  the  target  T (i)  anymore. 
If  the  engagement  has  failed,  then  a  branch  where  other  engagements  are  planned  for 
T (i)  is  executed.  All  these  conditional  branches  reflect  contingent  plans  that  are  very 
important  since  the  outcomes  of  the  engagements  are  uncertain.  Notice  that  without 
conditional  branches,  the  time  horizon  of  the  plan  would  be  very  limited  and  it  would 
be  needed  to  re-plan  each  time  an  engagement  fails.  The  latter  can  take  a  long  time, 
thus  causing  problems  for  the  subsequent  engagements. 

The  Holistic  Re-engagement  Planner  uses  the  following  rules 

(a)  the  closest  targets  are  engaged  first;  (threats  are  ranked  based  on  their  distance 
from  ownship,  the  closest  threat  has  rank  1) 

(b)  a  SAM  has  priority  over  the  57  mm  gun  to  engage  a  target 

(c)  the  CIWS  engages  whenever  possible 
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(d)  the  number  of  re-engagements  of  a  target  is  maximized 

4.  Tabu  Planner  -  Like  the  Holistic  Re-engagement  Planner,  the  Holistic  Tabu  Planner 
works  with  a  decision  tree  to  execute  the  plan,  and  it  considers  all  targets  for  the  plan. 
However  it  does  not  use  the  SAM  re-engagements,  but  instead  uses  Tabu  heuristics  for 
improving  the  quality.  An  initial  tree  is  first  created,  and  then  a  Tabu  search-based 
heuristics  is  used  to  improve  it.  The  initial  tree  is  improved  by  a  Tabu  search  [13] 
through  the  removal  or  addition  of  defence  actions,  followed  by  update  operations 
aimed  at  maintaining  the  consistency  of  the  plan. 

Tabu  search  is  based  on  an  iterative  neighbourhood  search  method  where  modifica¬ 
tions  to  the  current  solution  that  degrade  the  solution  value  are  admissible.  The 
latter  move  allows  the  method  to  escape  from  local  optima  (as  opposed  to  a  pure 
local  search  approach).  To  avoid  cycling,  a  short-term  memory,  known  as  the  Tabu 
list,  stores  previously  visited  solutions  or  components  of  previously  visited  solutions. 
It  is  then  forbidden  or  “Tabu”  to  come  back  to  these  solutions  for  a  certain  number 
of  iterations. 

3.2.2  Tools 

1.  Naval  Defence  Simulator  (NDS)  -  A  simulator  has  been  developed  during  this 
project  to  allow  for  large  amount  of  tests  of  the  investigated  concept  through  various 
scenarios.  The  Naval  Defence  Simulator  (NDS)  shown  on  Figure  2  allows  specific  tests 
to  be  replicated  as  many  times  as  desired,  which  is  obviously  impossible  to  match  on 
real-life  systems.  With  low  costs  compared  to  real-life  demonstrations,  this  allows 
to  develop,  implement,  validate  and  compare  a  broad  range  of  concepts.  Another 
advantage  of  having  a  simulator  is  that  it  allows  us  to  focus  on  particular  aspects 
of  the  C2  process.  In  the  project,  the  focus  has  been  on  resource  allocation  and 
coordination,  the  situation  analysis  problem  has  not  been  treated.  The  NDS  design 
and  implementation  were  driven  by  two  major  concerns:  extendibility  and  reusability. 
More  details  on  NDS  test-bed  can  be  found  in  [14,  15]. 

2.  Tactics  Generation  and  Evaluation  Tool  -  In  order  to  simulate  the  naval  defence 
environment,  a  test  bed  has  been  developed  (see  Figure  3).  Threats  appear  around 
the  ship  and  are  detected  (window  in  top  on  the  right).  Then,  the  decision  tree 
is  constructed  taking  into  account  the  different  constraints  (window  in  top  on  the 
left).  The  best  compromise  strategy  is  presented  in  the  window  in  bottom  on  the 
left.  In  bottom  on  the  right,  information  is  given  concerning  the  action  or  the  state 
of  the  nature  that  is  pointed.  Decision  tree,  as  well  as  generated  threats,  can  be 
saved  and  downloaded.  Myopic  (single-stage)  method  and  decomposition  method 
were  implemented  using  the  weighted  sum,  lexicographic,  TOPSIS  12  and  dominance 
as  Multi-Criteria  Decision  Analysis  (MCDA)  methods  [16,  17]. 

12.  Technique  for  Order  Preference  by  Similarity  to  Ideal  Solution. 
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Figure  2:  The  Naval  Defence  Simulator 


Figure  3:  Tactics  Generation  and  Evaluation  Tool 


DRDC  Valcartier  TR  2009-300 


19 


3.3  Publications 

The  work  conducted  under  llbrn  project  generated  several  publications,  ranging  from 
DRDC  Technical  reports  to  Master  thesis.  The  following  gives  an  exhaustive  list  of  these 
publications.  Among  other  information,  the  section  presents  the  abstract  of  the  different 
publications.  For  few  publications,  these  abstracts  are  modified  to  avoid  redundancy  in  the 
text.  The  modifications  concern  mainly  the  introductory  text  that  presents  the  context. 

3.3.1  Technical  Reports 

Six  (6)  DRDC  Technical  reports  were  produced  under  llbrn. 

1.  Multi- Agent  Coordination  Techniques  for  Naval  Tactical  Combat  Resources 
Management  [15]. 

Abstract  -  This  report  presents  a  review  of  agent  and  multi- agent  coordination  ap¬ 
proaches.  Theoretical  basis  of  distributed  planning  in  multi- agent  systems  is  intro¬ 
duced  and  coordination  mechanisms  are  described.  Multi-agent  approaches  are  used 
to  address  the  coordination  problems  for:  1)  hardkill/softkill,  2)  weapons  deploy¬ 
ment/ship  navigation,  and  3)  multi-ship  positioning  and  operations.  Results  of  the 
implementation  and  test  of  different  algorithms  for  these  combat  resource  coordina¬ 
tion  problems,  in  naval  engagements,  are  presented  and  discussed. 

2.  Combat  resource  allocation  planning  in  naval  engagements  [14]. 

Abstract-  This  report  presents  a  review  of  agent  and  multi-agent  planning  approaches. 
Theoretical  basis  of  agent  and  multi-agent  systems  are  introduced  and  planning  prob¬ 
lems  are  described.  The  results  of  the  implementation  and  test  of  different  algorithms 
for  hardkill  and  softkill  combat  resource  allocation,  in  naval  engagements,  are  pre¬ 
sented  and  discussed. 

3.  Single  Ship  Resource  Allocation  in  Above  Water  Warfare:  A  Multiple 
Criteria  Dynamic  Decision  Making  Approach  [17]. 

Abstract  -  This  report  is  concerned  with  dynamic  and  multi-criteria  decision-making 
problems  in  naval  engagement  context.  The  objective  is  to  identify  best  compromise 
strategies  for  weapons  allocation  against  a  set  of  incoming  threats  over  a  given  period 
of  time,  in  order  to  maximize  the  survivability  and  security  of  the  ship.  In  this  study, 
the  weapon  management  problem  is  represented  with  multiple  criteria  decision  tree 
and  solved  using  two  different  solving  approaches:  1)  the  Multiple  Criteria  Myopic 
approach  (MCM  approach),  which  deals  only  with  the  multiple  criteria  aspect;  and  2) 
the  Multiple  Criteria  Decomposition  approach  (MCD  approach),  which  considers  both 
the  multiple  criteria  and  the  dynamic  aspect  of  the  decision  problem.  The  empirical 
results  show  that  the  MCD  approach  is  always  superior  to  the  MCM  approach.  This 
report  argues  for  more  research  work  in  order  to  validate  the  results  with  subject 
matter  experts  and  contribute  to  the  development  of  tactics  for  highly  complex  and 
dynamic  decision-making  problems  like  weapon  management  of  the  Halifax  Class 
Frigate  facing  multiple  threats. 
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4.  Impact  of  the  Tactical  Picture  Quality  on  the  Fire  Control  Radar  Search- 
Lock- on  Time  [18] 

Abstract  -  Data  fusion  is  suitable  to  a  broad  range  of  decision  support  applications. 
To  allow  copping  with  a  larger  class  of  problems  and  contexts,  data  fusion  gains  to  be 
adaptive.  Adaptation  in  data  fusion  corresponds  to  Level  4  of  the  JDL  model,  also 
referred  to  as  process  refinement.  The  Decision  Support  Systems  (DSS)  Section  at 
Defence  Research  &  Development  Canada  (DRDC)  -  Valcartier  has  initiated  research 
activities  aiming  at  developing  and  demonstrating  advanced  concepts  of  adaptive 
data  fusion  that  could  apply  to  the  current  Halifax  and  Iroquois  Class  Command  & 
Control  Systems  (CCS),  as  well  as  their  possible  future  upgrades,  in  order  to  improve 
their  performance  against  the  predicted  future  threat.  This  document  gives  a  brief 
description  of  the  adaptive  data  fusion  concepts.  It  also  presents  a  new  Measure 
Of  Effectiveness  (MOE)  that  serves  as  an  adaptation  trigger  in  the  target-tracking 
problem  in  the  maritime  Above  Water  Warfare  (AWW)  applications.  The  proposed 
MOE  uses  the  search  and  lock-on  time  of  the  Fire  Control  Radar  (FCR)  and  aims  at 
establishing  and  quantifying  the  effect  of  the  quality  of  the  Maritime  Tactical  Picture 
(MTP)  on  the  diminution  of  the  battle  space  size  and  the  reaction  time.  Besides 
adaptation  of  the  sensing  and  processing  operation,  this  MOE  allows  addressing  the 
trade-off  finding  problem  between  the  time  dedicated  to  the  tracking  with  surveillance 
radars  versus  the  time  spent  in  search  and  lock-on  with  FCR. 

5.  Sensor  Management  in  the  Context  of  the  Integration  of  Sensors  and 
Weapons  [19]. 

Abstract  -  The  sensing  resources  represent  an  important  source  of  information  on 
which  the  Command  &  Control  (C2)  process  bases  most  of  its  reasoning.  Therefore, 
a  major  prerequisite  to  the  success  of  the  whole  C2  process  is  the  effective  use  of 
these  scarce  and  costly  resources.  This  is  the  problem  of  sensor  management  that 
has  to  do  with  how  best  to  manage,  coordinate  and  organize  the  use  of  sensing  re¬ 
sources  in  a  manner  that  synergistically  improves  the  process  of  data  acquisition  and 
ultimately  those  of  perception  and  comprehension,  i.e.,  the  situation  awareness  of 
the  decision  maker.  Conscious  of  the  important  role  sensor  management  has  to  play 
in  modern  Command  and  Control  systems,  the  Situation  Analysis  Support  Systems 
(SASS)  Group  of  the  Decision  Support  Systems  (DSS)  Section  at  Defence  Research 
&  Development  Canada  (DRDC)  -  Valcartier  has  studied  advanced  sensor  manage¬ 
ment  concepts  and  applications,  to  increase  the  survivability  of  the  current  Halifax 
and  Iroquois  Class  ships,  as  well  as  their  possible  future  upgrades.  The  objective  of 
the  reported  work  is  twofold  i)  to  present,  in  detail,  the  different  sensor  management 
problems  and  the  requirements  for  their  solution  ii)  to  demonstrate,  through  the  tar¬ 
get  tracking  application,  the  benefits  that  can  be  gained  by  the  adaptive  management 
of  the  available  sensors. 

3.3.2  Contract  Reports 

The  following  Three  (2)  contract  reports  were  produced  as  part  of  the  project  activities. 
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1.  Real-Time  Planning  and  Coordinating:  Naval  Resource  Management  Pro¬ 
cess  Description  [20]. 

Abstract  -  This  report  presents  a  description  of  the  Naval  Resource  Management  (RM) 
processes  for  a  single-platform  during  Naval  operations,  but  is  not  focussed  on  any 
specific  set  of  resources  or  system  architecture.  RM  was  defined  and  situated  in  the 
context  of  overall  Command  and  Control  (C2)  operations.  Emphasis  was  placed  on 
demonstrating  the  unique  role  played  by  RM  activities  in  light  of  Boyd’s  ’’Observe- 
Orient-Decide-Act”  (OODA)  Loop  model  of  C2,  and  on  the  relationship  of  RM  to 
the  functionalities  of  the  Joint  Directors  of  Laboratories  (JDL)  model  of  Information 
Fusion.  All  of  the  relevant  a  priori  and  dynamic  types  of  information  that  are  needed 
as  inputs  to  RM  processes  or  produced  as  a  consequence  of  that  processing  are  listed 
and  briefly  described.  Figures  are  presented  that  show  the  decomposition  of  RM 
into  its  constituent  fundamental  processes.  The  figures  also  demonstrate  some  of  the 
relationships  and  the  general  flow  between  processes.  Finally,  an  examination  of  each 
of  the  RM  processes  provides  a  brief  description  of  these  processes  and  identifies  the 
input/output  information  types  that  are  relevant  to  them. 

2.  Real-Time  Planning  and  Coordinating:  Final  Report  [21]. 

Abstract  -  This  report  presents  the  results  of  a  collaborative  project  that  investigated 
real-time  planning  and  coordinating  techniques  for  naval  Resource  Management  (RM) . 
A  literature  review  on  RM  planning  and  coordinating  techniques  was  conducted.  Met¬ 
rics  for  the  performance  evaluation  of  RM  techniques  were  investigated  and  developed. 
Five  types  of  algorithms  for  RM  were  identified  as  the  most  promising  for  further  de¬ 
velopment,  implementation  and  performance  evaluation:  Reflex,  Rollout,  Reinforce¬ 
ment  Learning  (specifically  Q-Learning),  Labelled  Real-Time  Dynamic  Programming 
(LRTDP),  and  Markovian  Constraint  Satisfaction  Process  (MaCSP)  (specifically,  a 
version  known  as  Focused  Real-Time  Dynamic  Planning  (FRTDP)).  The  Ship  Air  De¬ 
fence  Model  (SADM)  formed  the  basis  of  the  test  bed  used  to  develop  algorithms  and 
conduct  performance  evaluation.  The  test  scenarios  utilized  various  types  of  missile 
threats  coming  from  any  azimuth,  with  varying  start  times  and  ranges.  Ranking  the 
algorithms  from  best  to  worst,  for  platform  survival  the  order  was  LRTDP/FRTDP 
(similar),  Reflex,  Rollout,  and  Q-Learning.  For  planning  time,  the  order  was  Q- 
Learning,  Rollout,  FRTDP,  and  LRTDP.  Recommended  enhancements  for  each  of 
the  implemented  RM  algorithms  are  provided.  Finally,  observed  defects  and  anoma¬ 
lies  in  SADM  are  reported,  and  recommendations  for  modifications  and  upgrades  to 
SADM  to  support  further  RM  research  are  provided. 

3.  Tactical  Planning  and  Response  Management:  Investigating  a  Cognitive 
Work  Analysis  approach  to  the  development  of  support  concepts  [22]. 

Abstract  -  Cognitive  Work  Analysis  is  an  emerging  framework  for  designing  tools  to 
support  cognitively  demanding  work  in  complex  dynamic  environments.  To  inves¬ 
tigate  its  application  in  Naval  Command  and  Control,  DRDC  Atlantic  undertook 
a  Cognitive  Work  Analysis  of  tactical  planning  and  response  management  activities 
aboard  the  HALIFAX  Class  frigate,  focusing  on  the  job  of  the  Operations  Room  Of¬ 
ficer.  Four  interview  sessions  were  conducted  with  Subject  Matter  Experts  using  the 
Critical  Decision  Method.  The  data  collected  was  collated  into  a  chronological  or- 
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der  and  analyzed  according  to  two  steps  of  the  analysis  framework:  a  Work  Domain 
Analysis  and  a  Control  Task  Analysis.  The  first  step  mapped  the  activities  described 
by  the  experts  to  an  Abstraction-Decomposition  Space,  itself  comprised  of  an  Ab¬ 
straction  Hierarchy  and  a  Part- Whole  Decomposition.  The  second  step  mapped  each 
activity  to  a  Decision  Ladder  template.  A  variety  of  design  seeds  were  then  identi¬ 
fied  based  on  these  analyses.  Each  design  seed  linked  knowledge  elicitation  and  work 
analysis  outputs  directly  to  specific  design  hypotheses  for  work  support.  A  subset 
of  these  seeds  was  developed  further  into  a  tactical  planning  and  response  manage¬ 
ment  support  concept  for  presentation  to  a  new  set  of  Subject  Matter  Experts.  Based 
on  the  feedback  received,  the  support  concept  was  refined.  This  report  outlines  the 
data  collection  and  analysis  activities  undertaken.  It  also  details  the  identification  of 
design  seeds  from  the  various  analyses  conducted  and  the  coalescence  of  the  design 
seeds  into  an  integrated  support  concept.  Full  details  of  the  scenarios  described  by 
Subject  Matter  Experts,  the  analysis  results,  the  design  seeds  and  the  visualization 
of  the  support  concept  are  provided  in  the  annexes. 

3.3.3  Book  Chapters 

Two  (2)  book  chapters  were  published  based  on  the  work  conducted  under  llbrn. 

1.  Multiple  Criteria  Dynamic  Allocation  of  Shipboard  Weapons  [16]. 

Abstract  -  This  paper  is  concerned  with  decision-making  in  naval  operations.  Our 
scenario  involves  air  threats  directed  toward  a  ship  that  has  to  maximize  its  surviv¬ 
ability  by  managing  its  weapons.  This  dynamic  decision-making  problem  is  modeled 
using  multi-criteria  decision  trees.  A  dynamic  approach  is  then  proposed  to  solve  the 
problem.  This  approach  is  based  on  the  decomposition  of  the  tree  and  provides  the 
best  compromise  strategies.  The  results  it  yields  are  compared  with  those  provided 
by  a  myopic  approach,  where  the  dynamic  aspect  of  the  problem  is  simply  ignored. 

2.  Resource  Allocation  in  Time-Constrained  Environments:  The  Case  of 
Frigate  Positioning  in  Anti- Air  Warfare  [12]. 

Abstract  -  Maritime  environments  are  known  to  be  very  complex  environments  with 
tight  real-time  constraints  where  it  is  very  difficult  to  manage  resource  allocation. 
This  is  the  case,  for  example,  for  a  frigate  which  must  position  itself  in  order  to  use 
its  resources  the  most  effectively  possible  to  increase  its  chances  of  survival  when  the 
time  of  air  raids  comes.  Under  such  very  hard  constraints,  it  can  often  happen  that 
the  commander  makes  errors  because  of  the  complexity  of  the  environment  or  the 
stress  which  the  situation  can  generate.  We  propose  here  to  implement  a  decision-aid 
system  which  suggests  the  position  that  the  frigate  must  take.  We  start  by  giving  a 
heuristic  which  evaluates  the  effectiveness  of  a  position  according  to  the  threats  found 
in  the  environment.  Then,  we  propose  an  algorithm  which  treats  all  the  possible 
rotations  and  suggests  the  best  regarding  a  given  situation.  Finally,  we  expose  the 
results  of  our  experiments  and  we  comment  on  them. 
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3.3.4  Journal  Articles 


The  results  of  the  project  activities  yielded  the  three  (3)  following  journal  publications. 

1.  Target  Engageability  Improvement  through  Adaptive  Tracking  [23], 

Abstract  -  This  paper  addresses  the  joint  problem  of  target  engageability  assessment 
and  engageability  improvement  in  naval  Anti- Air  Warfare  operations.  An  integrated 
approach  that  aims  to  minimize  the  detect-to-engage  sequence  is  proposed.  It  uses 
an  estimation  of  the  search-to-lock-on  time  of  the  fire  control  radar  to  evaluate  the 
engageability  of  targets.  The  latter  is  then  improved  through  the  control  of  track¬ 
ing  operations.  Weapons  assignment  process  and  the  resulting  engagement  plan  are 
adjusted  based  on  the  results  of  both  the  assessment  and  the  improvement  of  the  en¬ 
gageability.  A  quantitative  evaluation  of  the  proposed  approach  was  performed  using 
a  simulation  and  performance  evaluation  environment  developed  at  Defence  Research 
and  Development  Canada  -  Valcartier.  Although  simple  sensors  and  weapons  models 
used  in  the  presented  work,  encouraging  results  were  obtained  with  scenarios  involv¬ 
ing  generic  supersonic  Anti-Ship  Missiles.  In  such  scenarios,  the  proposed  adaptive 
tracking  strategy  was  able  to  provide  timely  engagements  compared  to  a  conventional 
engagement  strategy. 

2.  A  Distributed  Intelligent  Tactical  Sensor  Management  System  [24]. 

Abstract  -  In  this  paper,  we  report  on  the  project  initiated  by  Defense  R&D  Canada 
-  Valcartier  that  is  intended  as  a  vehicle  to  assess  holonic  control  as  a  means  of 
improving  tactical  sensor  management  for  distributed  military  surveillance  operations. 
Three  levels  of  sensor  management  are  considered:  sensor,  platform,  and  group.  The 
proposed  design  is  used  to  develop  a  simulation  using  a  military  scenario  in  which  the 
holonic  control  system  is  employed  in  the  sensor  management  role.  The  results  of  this 
simulation  are  presented. 

3.  Multiagent  Coordination  Techniques  For  Complex  Environments:  The 
Case  of  a  Fleet  of  Combat  Ships  [25]. 

Abstract  -  The  use  of  agent  and  multiagent  techniques  to  assist  humans  in  their  daily 
routines  has  been  increasing  for  many  years,  notably  in  Command  and  Control  (C2) 
systems.  In  this  context,  we  propose  using  multiagent  planning  and  coordination  tech¬ 
niques  for  resources  management  in  real-time  C2  systems.  The  particular  problem  we 
studied  is  the  design  of  a  decision  support  for  Anti-Air  Warfare  (AAW)  on  combat 
ships.  In  this  paper,  we  refer  to  the  specific  case  of  several  combat  ships  defending 
against  incoming  threats  and  where  coordination  of  their  respective  resources  is  a 
complex  problem  of  capital  importance.  Efficient  coordination  mechanisms  between 
the  different  combat  ships  are  then  important  to  avoid  redundancy  in  engagements 
and  inefficient  defence  caused  by  the  conflicting  actions.  To  this  end,  we  present 
four  different  coordination  mechanisms  based  on  task  sharing.  Three  of  these  mecha¬ 
nisms  are  communication-based:  central  coordination,  contract  Net  coordination  and 
~Brown  coordination,  while  the  last  one  is  zone  defence  coordination  and  is  based  on 
conventions.  Finally,  we  expose  the  results  obtained  while  simulating  these  various 
mechanisms. 
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3.3.5  Conference  Proceedings 

The  work  performed  under  llbrn  was  presented  in  different  scientific  events,  including 

conferences,  symposia,  and  workshops.  Below  is  given  a  list  of  the  most  relevant  ones. 

1.  Fire  Control-Based  Adaptation  in  Data  Fusion  Applications  [26]. 

Abstract  -  In  military  Command  &  Control  applications,  the  information  quality  re¬ 
quirements  are  very  context-dependent  and  seldom  predefined.  This  leaves  much  room 
for  adaptation.  In  this  paper,  the  duration  of  the  search  &  lock-on  operations  of  the 
fire  control  radar  is  estimated  and  used  as  an  adaptation  trigger.  The  proposed  esti¬ 
mation  process  aims  at  establishing  a  quantitative  relationship  between  the  quality  of 
the  tactical  picture  and  the  reaction  time  available  for  decision-making.  Based  on  the 
target’s  time  of  flight,  the  defensive  weapon  properties,  and  the  desired  range  of  in¬ 
terception,  admissible  operational  conditions  and  constraints  for  the  fire  control  radar 
are  derived  to  allow  the  weapon  system  to  achieve  its  planned  interception.  These 
conditions  and  constraints  are  re-expressed  in  terms  of  tracking  quality  requirements. 
Then,  adaptation  mechanisms  are  used  to  select  and  tune  the  tracking  algorithms 
and/or  manage  sensors  in  order  to  meet  those  requirements. 

2.  A  Q-decomposition  and  Bounded  RTDP  Approach  to  Resource  Alloca¬ 
tion  [27], 

Abstract  -  This  paper  contributes  to  solve  effectively  stochastic  resource  allocation 
problems  known  to  be  NP-Complete.  To  address  this  complex  resource  management 
problem,  a  Q-decomposition  approach  is  proposed  when  the  resources  which  are  al¬ 
ready  shared  among  the  agents,  but  the  actions  made  by  an  agent  may  influence  the 
reward  obtained  by  at  least  another  agent.  The  Q-decomposition  allows  to  coordinate 
these  reward  separated  agents  and  thus  permits  to  reduce  the  set  of  states  and  actions 
to  consider.  On  the  other  hand,  when  the  resources  are  available  to  all  agents,  no 
Q-decomposition  is  possible  and  we  use  heuristic  search.  In  particular,  the  bounded 
Real-time  Dynamic  Programming  (bounded  RTDP)  is  used.  Bounded  RTDP  con¬ 
centrates  the  planning  on  significant  states  only  and  prunes  the  action  space.  The 
pruning  is  accomplished  by  proposing  tight  upper  and  lower  bounds  on  the  value 
function. 

3.  Tight  bounds  for  a  stochastic  resource  allocation  algorithm  using  marginal 
revenue  [28]. 

Abstract  -  This  paper  contributes  to  solve  effectively  stochastic  resource  allocation 
problems  known  to  be  NP-Complete.  To  address  this  complex  resource  management 
problem,  previous  works  on  pruning  the  action  space  of  real-time  heuristic  search 
is  extended.  The  pruning  is  accomplished  by  using  upper  and  lower  bounds  on  the 
value  function.  This  way,  if  an  action  in  a  state  has  its  upper  bound  lower  than  the 
lower  bound  on  the  value  of  this  state,  this  action  may  be  pruned  in  the  set  of  possible 
optimal  actions  for  the  state.  This  paper  extends  this  previous  work  by  proposing  tight 
bounds  for  problems  where  tasks  have  to  be  accomplished  using  limited  resources.  The 
marginal  revenue  bound  proposed  in  this  paper  compares  favourably  with  another 
approach  which  proposes  bounds  for  pruning  the  action  space. 
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4.  A  Decomposition  Real-time  Dynamic  Programming  Approach  to  Resource 
Allocation  [29]. 

Abstract  -  This  paper  contributes  to  solve  effectively  stochastic  resource  allocation 
problems  known  to  be  NP-Complete.  To  address  this  complex  resource  management 
problem,  the  merging  of  two  approaches  is  made:  The  Q-decomposition  model,  which 
coordinates  reward  separated  agents  through  an  arbitrator,  and  the  Labeled  Real- 
Time  Dynamic  Programming  (LRTDP)  approaches  are  adapted  in  an  effective  way. 
The  Q-decomposition  permits  to  reduce  the  set  of  states  to  consider,  while  LRTDP 
concentrates  the  planning  on  significant  states  only.  As  demonstrated  by  the  experi¬ 
ments,  combining  these  two  distinct  approaches  permits  to  further  reduce  the  planning 
time  to  obtain  the  optimal  solution  of  a  resource  allocation  problem. 

5.  An  Efficient  Model  for  Dynamic  and  Constrained  Resource  Allocation 
Problems  [30]. 

Abstract  -  Dynamic  constraint  satisfaction  is  a  useful  tool  for  representing  and  solving 
sequential  decision  problems  with  complete  knowledge  in  dynamic  world  and  partic¬ 
ularly  constrained  resource  allocation  problems.  However,  when  resources  are  unre¬ 
liable,  this  framework  becomes  limited  due  to  the  stochastic  outcomes  of  the  assign¬ 
ments  chosen.  On  the  contrary,  Markov  Decision  Processes  (MDPs)  handle  stochastic 
outcomes  of  unreliable  actions,  but  their  complexity  explodes  when  using  state-defined 
constraints.  We  thus  propose  an  extension  of  the  MDP  framework  so  as  to  represent 
constrained  and  stochastic  actions  in  sequential  decision  making.  The  basis  of  this 
extension  consists  in  modeling  the  evolution  of  a  dynamic  constraint  network  by  a 
MDP.  We  first  study  the  complexity  of  the  problem  of  finding  an  optimal  policy  for 
this  model  and  then  we  propose  an  algorithm  for  solving  it.  Comparison  to  standard 
MDP  shows  that  this  framework  noticeably  improves  policy  computation. 

6.  R-FRTDP:  A  Real-Time  DP  Algorithm  with  Tight  Bounds  for  a  Stochastic 
Resource  Allocation  Problem  [31]. 

Abstract  -  Resource  allocation  is  a  widely  studied  class  of  problems  in  Operation  Re¬ 
search  and  Artificial  Intelligence.  Specially,  constrained  stochastic  resource  allocation 
problems,  where  the  assignment  of  a  constrained  resource  do  not  automatically  imply 
the  realization  of  the  task.  This  kind  of  problems  are  generally  addressed  with  MDPs. 
In  this  paper,  we  present  efficient  lower  and  upper  bounds  in  the  context  of  a  con¬ 
strained  stochastic  resource  allocation  problem  for  a  heuristic  search  algorithm  called 
Focused  Real  Time  Dynamic  Programming  (FRTDP).  Experiments  show  that  this 
algorithm  is  relevant  for  this  kind  of  problems  and  that  the  proposed  tight  bounds 
reduce  the  number  of  backups  to  perform  comparatively  to  previous  existing  bounds. 

7.  Argumentation-based  Decision  Support  in  Naval  Command  and  Control  [32]. 

Abstract  -  Threat  Evaluation  and  Weapons  Assignment  (TEWA),  a  process  which  is 
at  the  heart  of  tactical  naval  Command  &  Control  (C2)  process,  comprises  a  number  of 
operations  that  must  be  performed  under  time  and  resource  constraints.  This  article 
discusses  the  challenges  of  decision  making  in  this  context,  and  more  particularly  the 
critical  issue  of  target  engagement,  and  shows  how  this  process  can  be  supported  by  an 
argumentation-based  Decision  Support  System  (DSS).  It  is  shown  how  the  information 


DRDC  Valcartier  TR  2009-300 


gathered  and  analyzed  during  the  execution  of  the  engageability  assessment,  defined 
and  formalized  for  the  purpose  of  the  paper,  can  be  exploited  by  an  argumentation 
module.  Based  on  a  dialectical  model  and  affording  both  proactive  and  reactive 
interaction  modes,  the  module  enables  the  DSS  to  anticipate  and  respond  to  the 
operator’s  objections  to  its  recommendations,  and  thus  substantially  enhance  the 
accuracy  of  its  argumentation  in  a  time-constrained  decision  support  context. 

8.  The  Application  of  Holonic  Control  to  Tactical  Sensor  Management  [33], 
Abstract  -  In  this  paper,  we  report  on  the  project  initiated  by  Defence  R&D  Canada 
-  Valcartier  that  is  intended  as  a  vehicle  to  assess  holonic  control  as  a  means  of 
improving  tactical  sensor  management  for  distributed  military  surveillance  operations. 
Three  levels  of  sensor  management  are  considered:  sensor,  platform,  and  group.  The 
general  holonic  architecture  and  the  individual  holons  at  each  level  are  described  as 
well  as  our  next  steps  towards  the  implementation  of  a  simulation  of  a  typical  scenario. 

9.  Adaptation  Hierarchy  for  Data  Fusion  and  Sensor  Management  Applica¬ 
tions  [34], 

Abstract  -  By  reducing  uncertainty  in  the  existing  pieces  of  information  and  providing 
means  to  infer  about  the  missing  pieces,  data  fusion  supports  the  decision-makers  in 
compiling  and  analyzing  a  representation  of  the  problem  of  interest.  Data  fusion  has 
often  been  portrayed  as  a  sequential  process  that  simply  combines  pieces  of  infor¬ 
mation  in  a  purely  open-loop  mode.  Nevertheless,  to  cope  with  changing  objectives, 
environments  and  constraints,  fusion  systems  need  an  active  feedback,  or  adaptation. 
This  article  presents  part  of  our  research  activities  that  aim  at  defining,  developing, 
and  demonstrating  adaptation  concepts  in  data  fusion  and  sensor  management.  Hier¬ 
archy  of  adaptation  and  management  problems  are  defined  and  holonic  architecture 
is  identified  as  suitable  control  structure.  The  holonic  architecture  presents  a  superior 
choice  in  the  military  settings  because  it  is  naturally  hierarchical  and  recursive.  A 
holonic  architecture  maintains  the  chain  of  command,  is  robust  and  flexible,  and  its 
overall  behaviour  is  predictable. 

10.  An  Efficient  Resource  Allocation  Approach  in  Real-time  Stochastic  Envi¬ 
ronment  [35], 

Abstract  -  We  are  interested  in  contributing  to  solving  effectively  a  particular  type 
of  real-time  stochastic  resource  allocation  problem.  Firstly,  one  distinction  is  that 
certain  tasks  may  create  other  tasks.  Then,  positive  and  negative  interactions  among 
the  resources  are  considered,  in  achieving  the  tasks,  in  order  to  obtain  and  maintain 
an  efficient  coordination.  A  standard  Multiagent  Markov  Decision  Process  (MMDP) 
approach  is  too  prohibitive  to  solve  this  type  of  problem  in  real-time.  To  address  this 
complex  resource  management  problem,  the  merging  of  an  approach  which  considers 
the  complexity  associated  to  a  high  number  of  different  resource  types  (i.e.  Multiagent 
Task  Associated  Markov  Decision  Processes  (MTAMDP)),  with  an  approach  which 
considers  the  complexity  associated  to  the  creation  of  task  by  other  tasks  (i.e.  Acyclic 
Decomposition)  is  proposed.  The  combination  of  these  two  approaches  produces  a 
near-optimal  solution  in  much  less  time  than  a  standard  MMDP  approach. 

11.  A  Multiagent  Task  Associated  MDP  (MTAMDP)  Approach  to  Resource 
Allocation  [36]. 
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Abstract  -  We  are  interested  in  contributing  to  solving  effectively  a  specific  type  of 
real-time  stochastic  resource  allocation  problem,  which  is  known  as  NP-Hard,  of  which 
the  main  distinction  is  the  high  number  of  possible  interacting  actions  to  execute 
in  a  group  of  tasks.  To  address  this  complex  resource  management  problem,  we 
propose  an  adaptation  of  the  Multiagent  Markov  Decision  Process  (MMDP)  model 
which  centralizes  the  computation  of  interacting  resources.  This  adaptation  is  called 
Multiagent  Task  Associated  Markov  Decision  Process  (MTAMDP)  and  produces  a 
near-optimal  policy  in  a  much  lower  time  than  a  standard  MMDP  approach.  In  a 
MTAMDP,  a  planning  agent  computes  a  policy  for  each  resource,  and  are  coordinated 
by  a  central  agent.  MTAMDPs  enables  to  practically  solve  our  NP-Hard  problem. 

12.  Threat  Evaluation  and  Weapons  Allocation  in  Network-Centric  Warfare  [37], 

Abstract  -  The  concepts  of  threat  evaluation  and  weapons  allocation  (TEWA)  in  the 
defence  domain  have  traditionally  been  considered  from  the  single  platform  perspec¬ 
tive.  However,  with  the  current  trend  in  defence  towards  network-centric  warfare, 
that  is  the  linking  of  sensors,  engagement  systems  and  decision-makers  into  an  effec¬ 
tive  and  responsive  whole,  it  is  becoming  more  appropriate  to  view  these  concepts 
at  the  force  level.  One  approach  to  the  challenge  of  developing  force  level  TEWA 
functionality  is  to  regard  TEWA  as  a  dynamic  human  decision-making  process  aimed 
at  the  successful  exploitation  of  tactical  resources  (eg  sensors  and  weapons)  during 
the  conduct  of  command  and  control  activities.  In  this  paper,  the  results  of  taking 
this  approach  to  force  level  TEWA  through  the  application  of  the  applied  cognitive 
work  analysis  methodology  are  presented.  In  particular,  a  functional  abstraction  net¬ 
work  is  described  which  encapsulates  the  inferential  transformation  from  sensor  data 
acquisition  to  inferences  about  the  identification,  intent  and  level  of  threat  for  the 
given  entities  in  the  defence  environment.  Finally,  emerging  threat  evaluation  and 
weapons  allocation  concepts  in  network-centric  warfare  are  outlined  and  an  example 
is  given  to  illustrate  the  ideas  developed  within  the  paper. 

13.  Decomposition  Techniques  for  a  Loosely-Coupled  Resource  Allocation  Prob¬ 
lem  [5]. 

Abstract  -  We  are  interested  by  contributing  to  stochastic  problems  of  which  the  main 
distinction  is  that  some  tasks  may  create  other  tasks.  In  particular,  we  present  a  first 
approach  which  represent  the  problem  by  an  acyclic  graph,  and  solves  each  node 
in  a  certain  order  so  as  to  produce  an  optimal  solution.  Then,  we  detail  a  second 
algorithm,  which  solves  each  task  separately,  using  the  first  approach,  and  where  an 
on-line  heuristic  computes  the  global  actions  to  execute  when  the  state  of  a  task 
changes. 

14.  Multi-Platform  Coordination  in  Command  and  Control  [38], 

Abstract  -  The  use  of  agent  and  multi- agent  techniques  to  assist  human  in  its  daily 
routine  has  been  increasing  for  many  years,  notably  in  Command  and  Control  (C2) 
systems.  In  this  article,  we  focused  on  multi-agent  coordination  techniques  for  re¬ 
sources  management  in  realtime  C2  systems.  The  particular  problem  we  studied  is 
the  design  of  a  decision-support  for  anti-air  warfare  on  Canadian  frigates.  In  the 
case  of  the  several  frigates  defending  against  incoming  threats,  multi-agent  coordi¬ 
nation  is  a  complex  problem  of  capital  importance.  Better  coordination  mechanisms 
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are  important  to  avoid  redundancy  in  engagements  and  inefficient  defence  caused  by 
conflicting  actions.  We  present  different  task  sharing  coordination  mechanisms  with 
their  evaluation. 

15.  An  Agent-Based  Decision  Support  System  for  Naval  Anti-Air  Warfare:  An 
Exploration  Based  on  Simulation  [10]. 

Abstract  -  Maritime  environments  are  known  to  be  very  complex  environments  with 
tight  real-time  constraints  where  it  is  very  difficult  to  manage  resource  allocation. 
This  is  the  case,  for  example,  for  a  frigate  which  must  position  itself  in  order  to  use 
its  resources  the  most  effectively  possible  to  increase  its  chances  of  survival  when  the 
time  of  air  raids  comes.  Under  such  very  hard  constraints,  it  can  often  happen  that 
the  commander  makes  errors  because  of  the  complexity  of  the  environment  or  the 
stress  which  the  situation  can  generate.  We  propose  here  to  implement  a  decision-aid 
system  which  suggests  the  position  that  the  frigate  must  take.  We  start  by  giving  an 
heuristic  which  evaluates  the  effectiveness  of  a  position  according  to  the  threats  found 
in  the  environment.  Then,  we  propose  an  algorithm  which  treats  all  the  possible 
rotations  and  suggests  the  best  regarding  a  given  situation.  Finally,  we  expose  the 
results  of  our  experiments  and  we  comment  on  them. 

16.  A  Frigate  Movement  Survival  Agent-Based  Approach  [39], 

Abstract  -  The  position  of  a  frigate  to  face  some  threats  can  augment  its  survival 
chances  and  therefore  it  is  important  to  investigate  this  aspect  in  order  to  determine 
how  a  frigate  can  position  itself  during  an  attack.  To  achieve  that,  we  propose  a 
first  method  based  on  the  Bayesian  movement,  performed  by  a  learning  agent,  which 
determines  the  optimal  positioning  of  the  frigate  by  dividing  the  defense  area  into 
six  sectors  for  weapon  engagement  and  then,  it  makes  efficient  use  of  all  the  weapons 
available  by  using  the  sectors.  The  second  method  that  we  propose  is  called  Radar 
Cross-Section  Reduction  (RCSR)  movement  and,  it  aims  at  reducing  the  exposed  sur¬ 
face  of  the  frigate  to  incoming  threats  before  their  locking  phase  is  over.  Preliminary 
results  on  these  two  methods  are  presented  and  discussed.  Finally,  an  implementation 
of  a  meta-level  agent  which  would  make  efficient  use  of  both  complementary  methods 
is  suggested. 

17.  Sensor  Management  in  Command  &  Control  [40]. 

Abstract  -  The  sensing  resources  represent  an  important  source  of  information  on 
which  the  Command  &  Control  (C2)  process  bases  most  of  its  reasoning.  Therefore, 
a  major  prerequisite  to  the  success  of  the  whole  C2  process  is  the  effective  use  of 
these  scarce  and  costly  resources.  This  is  the  problem  of  sensor  management  that 
has  to  do  with  how  best  to  manage  and  coordinate  the  use  of  sensing  resources  to 
improve  data  acquisition  and  ultimately  perception  and  comprehension.  Conscious 
of  the  important  role  sensor  management  has  to  play  in  modern  C2  systems,  the  De¬ 
cision  Support  Systems  (DSS)  Section  at  Defence  Research  &  Development  Canada 
-  Valcartier  is  currently  studying  advanced  sensor  management  concepts  and  appli¬ 
cations,  to  increase  the  survivability  of  the  current  Halifax  and  Iroquois  Class  ships, 
as  well  as  their  possible  future  upgrades.  The  objective  of  the  reported  part  of  this 
study  is  twofold  i)  to  present  the  sensor  management  problem  and  the  requirements 
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for  its  solution  ii)  to  demonstrate,  through  a  tracking  application,  the  benefits  that 
can  be  gained  by  the  closed-loop  management  of  the  sensors. 

18.  A  method  to  optimize  ship  maneuvers  for  the  coordination  of  hardkill  and 
softkill  weapons  within  a  frigate  [11]. 

Abstract  -  The  coordination  of  anti- air  warfare  hardkill  and  softkill  weapon  systems 
is  an  important  aspect  of  command  and  control  for  a  Frigate.  Since  the  effectiveness 
of  a  particular  weapon  varies  depending  on  the  orientation  of  the  Frigate  with  respect 
to  the  threats  faced,  a  key  element  of  the  coordination  process  is  to  maneuver  the 
Frigate  to  most  effectively  use  all  the  weapons  available.  This  paper  shows  that  the 
environment  surrounding  the  Frigate  can  be  divided  into  six  fundamental  sectors  for 
weapon  engagement.  The  method  to  determine  the  general  effectiveness  of  each  sector 
for  the  threats  faced  is  shown.  A  naive  Bayes  method  that  determines  the  optimal 
positioning  of  the  Frigate  to  most  effectively  use  the  hardkill  and  softkill  weapons  is 
presented.  Also  discussed  are  the  different  types  of  planners  that  were  investigated 
for  planning  engagements  for  the  hardkill  and  softkill  weapon  systems.  Preliminary 
results  comparing  and  rating  these  planners  are  shown,  both  with  and  without  the 
recommended  maneuvers. 

3.3.6  Theses 

Most  of  the  work  conducted  under  limb  was  achieved  in  collaboration  with  with  Cana¬ 
dian  industry  and  universities.  This  section  gives  the  list  of  the  graduated  students,  who 

contributed  the  this  work,  and  their  achievements. 

1.  Techniques  For  The  Allocation  Of  Resources  Under  Uncertainty  (Pierrick 
Plamondon,  Ph.D),  U.  Laval. 

Abstract  -  Resource  allocation  is  an  ubiquitous  problem  that  arises  whenever  limited 
resources  have  to  be  distributed  among  multiple  autonomous  entities  (e.g.,  people, 
companies,  robots,  etc).  The  standard  approaches  to  determine  the  optimal  resource 
allocation  are  computationally  prohibitive.  The  goal  of  this  thesis  is  to  propose  com¬ 
putationally  efficient  algorithms  for  allocating  consumable  and  non-consumable  re¬ 
sources  among  autonomous  agents  whose  preferences  for  these  resources  are  induced 
by  a  stochastic  process.  Towards  this  end,  we  have  developed  new  models  of  plan¬ 
ning  problems,  based  on  the  framework  of  Markov  Decision  Processes  (MDPs),  where 
the  action  sets  are  explicitly  parameterized  by  the  available  resources.  Given  these 
models,  we  have  designed  algorithms  based  on  dynamic  programming  and  real-time 
heuristic  search  to  formulate  allocations  of  resources  for  agents  evolving  in  stochastic 
environments.  In  particular,  we  have  used  the  acyclic  property  of  task  creation  to  de¬ 
compose  the  problem  of  resource  allocation.  We  have  also  proposed  an  approximative 
decomposition  strategy,  where  the  agents  consider  positive  and  negative  interactions 
as  well  as  simultaneous  actions  among  the  agents  managing  the  resources.  However, 
the  main  contributions  of  this  thesis  is  the  adoption  of  stochastic  real-time  heuristic 
search  for  a  resource  allocation.  To  this  end,  we  have  developed  an  approach  based 
on  Q-decomposition  with  tight  bounds  to  diminish  drastically  the  planning  time  to 
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formulate  the  optimal  policy.  These  tight  bounds  enable  to  prune  the  action  space 
for  the  agents.  We  show  analytically  and  empirically  that  our  proposed  approaches 
lead  to  drastic  (in  many  cases,  exponential)  improvements  in  computational  efficiency 
over  standard  planning  methods.  Finally,  we  have  tested  real-time  heuristic  search  in 
the  SADM  simulator,  a  simulator  for  the  resource  allocation  of  a  platform. 

2.  A  Method  to  solve  dynamic  and  multi-criteria  decision  problems  under 
uncertainty  (Anissa  Frini,  Ph.D),  U.  Laval. 

Abstract  -  The  general  objective  of  this  thesis  is  the  development  of  a  resolution 
method  for  multi-criteria  dynamic  decision-making  problems  under  uncertainty.  In 
particular,  this  thesis  answers  the  following  questions:  is  it  possible  to  combine  the 
decomposition  of  the  decision  tree  when  a  multi-criteria  decision  aid  method  is  used  to 
get  best  compromise  strategies?  What  are  conditions  that  guarantee  best  compromise 
strategies  via  decomposition?  How  could  we  select  the  multi-criteria  decision  aid 
method  that  will  be  used?  What  are  the  resolution  method  steps?  Can  we  apply 
the  proposed  method  for  concrete  applications  and  which  are  its  advantages  and 
its  limits?  To  answer  these  questions,  a  principle  of  decomposition  that  generalizes 
the  Bellman  decomposition  principle  to  multi-criteria  decision  trees,  is  stated.  This 
principle  leads  to  strategies  of  best  compromise  when  decomposition  conditions  are 
verified.  A  theorem,  which  specifies  these  conditions,  is  stated  and  proved.  When 
the  principle  of  decomposition  is  applied,  a  multi-criteria  decision  aid  method  is  used 
in  a  recursive  way  for  each  sub-problem.  The  choice  of  this  multi-criteria  decision 
aid  method  is  important.  It  will  be  made  considering  not  only  the  decomposition 
conditions  but  also  some  theoretical  and  pragmatic  considerations.  Afterwards,  the 
steps  of  the  resolution  method  are  proposed  for  two  cases.  At  first,  the  method  is 
presented  for  quantitative  criteria  and  probabilistic  uncertainty.  Then,  the  method  is 
presented  for  qualitative  criteria  and  possibilistic  uncertainty,  modeled  by  possibility 
distributions.  Finally,  the  proposed  method  is  applied  for  the  defense  of  a  military 
naval  ship.  This  application  aims  to  generate  all  feasible  defense  strategies  and  to 
select  best  compromise  ones.  Results  of  the  decomposition  method  are  compared 
with  a  myopic  method,  which  simply  ignores  the  dynamic  aspect  of  the  problem. 

3.  Ordonnancement  de  ressources  en  temps  reel  avec  contraintes  dynamiques 
dans  un  environnement  non  deterministe  (Olivier  Gagne,  Master),  U. 
Laval. 

Abstract  -  Military  problems  are  very  complex  and  they  can  be  solved  by  different 
artificial  intelligence  techniques.  In  this  thesis,  we  address  the  problem  of  weapon- 
targets  assignment  for  a  frigate.  To  defend  efficiently  the  ship,  we  have  to  analyze 
each  threat  and  determine  which  resource  assigns  against  it.  For  that  purpose,  we 
utilize  the  engageability  assessment  to  consider  different  characteristics;  useful  in  the 
resources  assignment.  To  this  end,  a  mathematical  model  named  Constraint  Satisfac¬ 
tion  Problem  (CSP)  is  employed.  This  framework  allows  formalizing  the  problem  to 
ensure  the  constraint  consistency  and  to  sort  threats  in  importance  order.  We  tried 
this  algorithm  on  different  types  of  weapon-target  assignment  problems.  Finally,  we 
demonstrate  the  advantage  of  engageability  assessment  on  the  weapon-target  assign¬ 
ment  problem  in  real  time  and  stochastic  environment. 
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4.  Coordination  de  plans  d’agents  Application  a  la  gestion  des  ressources 
d’une  fregate  (Sebastien  Paquet,  Master),  U.  Laval. 

Abstract  -  In  computer  science,  agents  and  multi-agent  systems  have  become  very 
popular  and  are  now  used  for  many  varied  applications.  Depending  on  the  target 
application,  the  agents  can  take  different  architectures.  These  architectures  can  be 
classified  as  follows:  reactive,  deliberative  and  hybrid.  In  many  cases,  the  agents 
do  not  exist  alone  in  their  environment  and  they  must  be  able  to  coordinate  their 
activities  in  order  to  attain  a  certain  goal.  They  must  be  able  to  interact  together  in 
such  a  way  that  conflicting  actions  are  avoided  and,  if  possible,  that  favors  positive 
interactions.  There  are  many  coordination  techniques  available,  such  as  the  use  of  an 
organizational  structure,  contract  allocation,  multi-agent  planning,  negotiation,  the 
use  of  engagements,  etc.  However,  under  certain  circumstances,  coordination  by  itself 
is  not  sufficient.  Sure  enough,  the  agents  must  not  only  coordinate  well,  they  have  to 
really  act  together,  in  other  words,  act  as  a  team.  The  difference  between  an  agent 
team  and  a  group  of  cooperating  agents  is  that  the  team  not  only  acts  in  order  to 
realize  a  certain  objective,  they  do  it  in  a  certain  “mindset” ,  with  the  knowledge  that 
they  are  part  of  a  team.  This  allows  a  greater  cohesion  in  the  team.  As  an  example, 
an  agent  might  help  another  if  it  realizes  that  the  other  needs  help  or  if  it  realizes  that 
the  task  could  be  accomplished  more  efficiently.  In  this  thesis,  multi-agent  systems 
and  agents  teams  (both  constituted  of  hybrid  agents)  are  put  to  use  in  the  design  of  a 
resource  management  system  for  a  Canadian  Halifax  patrol  frigate.  Ships  of  this  class 
feature  many  weapon  control  systems  which  must  be  coordinated  efficiently  in  order 
to  maximize  the  chance  of  survival  when  a  frigate  is  attacked  by  anti-ship  missiles. 

5.  A  Frigate  Survival  Approach  based  on  Real-time  Multi-agent  Planning 
(Pierrick  Plamondon,  Master),  U.  Laval. 

Abstract  -  Nowadays,  in  computer  sciences,  intelligent  agents  are  becoming  more  and 
more  popular.  Such  systems  offer  a  logical  approach  to  conceive  a  system  where 
diverse  modules  interact  to  solve  complex  real  world  problems.  Lockheed  Martin 
Canada  (LMC)  and  the  Defence  Research  and  Development  Canada  —  Valcartier 
(DRDC  Valcartier)  have  provided  a  real  world  multi-agent  application.  Our  main 
task  for  this  application  is  to  conceive  different  multi-agent  techniques  to  improve 
the  defensive  effectiveness  of  a  Halifax  class  Canadian  frigate  against  incoming  mis¬ 
sile  threats.  A  Halifax  class  frigate  uses  different  modules  that  interact  together  to 
defend  itself  and  it  is  necessary  to  propose  ways  to  optimize  the  coordination,  be¬ 
tween  them  in  order  to  increase  their  efficiency.  In  this  thesis,  we  will  focus  mainly  on 
multi-agent  planning.  We  first  present  some  theoretical  bases  on  planning,  different 
planning  approaches,  then  we  investigate  how  to  improve  the  effectiveness  of  defence 
for  the  frigate  using  different  strategies.  Finally,  we  provide  different  experimental 
results.  Our  experimental  results  show  that  1)  using  missile  re-engagement  against  in¬ 
coming  threats  is  more  efficient  with  few  threats  than  with  many  threats;  2)  a  central 
coordinator  coordination  technique  between  hardkill  and  softkill  agents  is  better  than 
with  two  other  coordination  techniques;  and  3)  a  Bayesian  approach  for  the  frigate 
positioning  enhances  the  frigate’s  chance  of  survival. 

6.  Real-time  Deliberative  Planning  (Martin  Soucy,  Master),  U.  Laval. 
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Abstract  -  For  several  years,  Defence  Research  &  Development  Canada  (DRDC)  and 
Lockheed  Martin  Canada  (LMC)  have  been  working  together  in  order  to  modernize 
the  Command  and  Control  system  (C2)  present  on-board  a  frigate.  The  purpose  of 
such  a  system  is  to  analyze  and  manage  a  considerable  flow  of  information  in  order 
to  carry  out  the  right  actions  in  response  to  a  given  situation.  Our  role  within  this 
project  is  at  the  decision-making  level.  We  must  study  the  possibilities  that  agent 
technology  can  offer  to  such  a  system.  In  this  thesis,  we  focused  our  efforts  on  the 
implications  of  real  time  planning  adapted  to  agents  and  multi- agent  systems.  For 
this  purpose,  we  will  first  of  all  present  the  theoretical  elements  of  real  time  planning 
in  agent  systems  and  the  paradigm  of  mobile  agents.  We  will  then  show  our  results 
concerning  two  approaches  to  carry  out  the  planning  of  an  agent  and  of  a  group  of 
frigate  agents. 

7.  Multi-Platform  Coordination  and  Resource  Management  in  Command  and 
Control  (Patrick  Beaumont,  Master),  U.  Laval. 

Abstract  -  The  use  of  agent  and  multi-agent  techniques  to  assist  humans  in  their  daily 
routines  has  been  increasing  for  many  years,  notably  in  Command  and  Control  (C2) 
systems.  This  thesis  is  situated  in  this  domain.  Precisely,  we  propose  to  use  multi¬ 
agent  planning  and  coordination  techniques  for  resource  management  in  real-time 
C2  systems.  The  particular  problem  we  studied  is  the  design  of  a  decision-support 
for  anti- air  warfare  on  Canadian  frigates.  In  the  case  of  several  frigates  defending 
against  incoming  threats,  multi-agent  coordination  is  a  complex  problem  of  capital 
importance.  Better  coordination  mechanisms  are  important  to  avoid  redundancy 
in  engagements  and  inefficient  defence  caused  by  conflicting  actions.  In  this  thesis, 
we  present  four  different  coordination  mechanisms  based  on  task  sharing.  Three  of 
these  mechanisms  are  based  on  communications:  central  coordination,  Contract  Net 
coordination  and  Brown  coordination,  while  the  zone  defence  coordination  is  based 
on  social  laws.  Finally,  we  expose  the  results  obtained  while  simulating  these  various 
mechanisms. 

8.  Real-time  planning  of  ship  position  in  response  to  anti-ship  missile  attack 
(Jean-Frangois  Morissette,  Master),  U.  Laval. 

Abstract  -  For  several  years,  Recherche  et  Developpement  pour  la  Defense  Canada 
(RDDC)  and  Lockheed  Martin  Canada  (LMC)  have  been  working  together  in  order  to 
modernize  the  Command  and  Control  system  (C2)  present  on  a  frigate.  The  purpose 
of  such  a  system  is  to  analyze  and  manage  a  considerable  flow  of  information  in  order 
to  carry  out  the  right  actions  in  response  to  a  given  situation.  Our  role  within  this 
project  is  at  the  decision-making  level.  We  must  study  the  possibilities  that  agent 
technology  can  offer  to  such  a  system.  In  this  thesis,  we  propose  a  formal  model  of 
interaction  between  the  various  resources  available  on  the  frigate  and  the  threats  it 
must  face.  We  also  propose  a  technique  of  positioning  and  an  approach  of  planning 
in  order  to  maximize  the  chances  of  survival  of  the  frigate.  Lastly,  we  present  the 
results  we  obtained  by  means  of  simulations. 
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4  Recommendations  for  Future  Work 


This  report  presents  a  summary  of  the  problems  tackled,  the  technological  approaches 
investigated,  and  the  achievements  realized  by  the  llbrn  Applied  Research  Project  (ARP) 
on  combat  resource  management. 

The  major  significance  of  this  project  lies  in  that  it  proposes  and  explores  some  powerful 
practical  and  theoretical  ideas  into  the  core  of  decision  support  and  automation  for  combat 
resource  management.  The  foundational  ideas  come  from  a  variety  of  areas,  including 
multi-agent  systems,  agent-based  planning,  distributed  and  multi-criteria  decision  making, 
man-machine  interaction,  coordination  between  agents,  cognitive  systems  engineering  and 
modeling  and  simulation.  The  work  conducted  under  llbrn  had  an  important  import 
impact  on  the  technologies/approaches  adopted  by  SISWS  TDP  for  the  development  of 
the  weapons  manager  and  by  INCOMMANDS  TDP  for  the  development  of  CORALS,  the 
combat  resource  management  capability  [41,  42]. 

The  project  covers  several  topics  and  problems,  yet  many  others  will  require  further  inves¬ 
tigation.  The  following  gives  a  list  of  areas/problems  where  such  investigation  is  required: 

1.  Constraint-based  target  engageability  assessment. 

2.  Dynamic  tasking  and  allocation  of  area  of  responsibility  to  defence  assets  (Task 
Group)  in  order  to  maximize  the  exploitation  of  the  available  reaction-time  to  defeat 
air  and  surface  (symmetric/asymmetric)  threats  ( e.g .,  small  boat /aircraft  attack). 

3.  Multi-platform  and  multi-environment  (Navy,  Air  Force,  Army)  weapons/weapons 
and  weapons/sensors  coordination  and  cooperation  problems  (e.g.,  to  support  the 
Army  and  the  Air  Force  to  defeat  land-based  threats  by  providing  Naval  Fire  Support). 

4.  Dynamic  information  gathering  concepts  to  effectively  assess  the  level  of  the  response 
success  (e.g.,  kill  assessment).  Requirements  will  need  to  be  gathered  for  surveil¬ 
lance  tasks  to  gain  reaction-time  and  increase  the  targets  engageability  and  objects 
discrimination  power. 

5.  Dynamic  allocation  and  coordination  of  sensing  resources  to  maximize  the  task-related 
value  of  the  collected  information  in  order  to  1)  Increase  the  reaction-time;  2)  Increase 
the  engageability  of  the  threats  by  providing  a  higher  discrimination  power  (weapon 
quality  information)  in  target-dense  environments;  and  3)  Minimize  the  yet  high  risk 
of  collateral  damages  (e.g.,  in  littoral  context). 

6.  Sensors  mode  control,  sensor  tasking,  and  sensor  coordination  algorithms  and  solu¬ 
tions  in  support  of  adaptive  information  fusion  concepts. 

7.  Coordination  of  platforms  deployment  for  a  synergetic  distributed  information  gath¬ 
ering. 
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